ADF-Git Integration

The Problem

I was playing around with Azure Data Factory and I saw it had Git integration. Awesome. I enabled it on one of my Azure Devops repositories, and away we went. It allowed me to branch and do my work there.

Then I hit publish. A new branch sprang up, adf_publish, so I did a git pull locally, switched to it, and everything in my repo disappeared. What the hell???

I wanted a visual view of my repo so I did a quick search and found gmaster. I installed it, pointed it at my repo and this is what came up.

What is this malarkey?

Now I don’t claim to know everything about Git or VCS’s, but this is utterly fascinating to me. What appears to be a branch that starts freely in space, with no branching from main or any other branch. I must know more!

The Investigation

So I took a look at that image, and immediately had three questions pop into my brain:

  • Is this dependent on the initial branch name (master vs main)?
  • What enables this mechanism?
  • Why does ADF do that?

I did an investigation into the first question. I created two Github Repos, one based on main, and one based on master as initial branch names. Then I created two Azure Data Factories, pointed them at the git repos, and worked through this tutorial, thanks to Cathrine Wilhelmsen (b|t). The moment I hit publish for the first time, I had my answer. It turns out, no, it’s not dependent on the intial branch name. Here is with main vs. master, and you can see they both have the same characteristic:

initial branch named master
initial branch named main

So the idea it was dependent on the repo was thrown out.

Git has the capability to create what is called an orphan branch. Here is me creating one, and then the effects:

git checkout –orphan dalehirt_publish
My safe little orphaned branch, all by it’s lonesome

Great. I can recreate easily what ADF is doing. So what is an orphaned branch good for? Here is direct from the git documentation:

So it’s actually good for publishing from a clean branch with no ties to the basic history. Fascinating. I found a great little script on how to create orphan branches and it appears the ADF people followed its recommendations.

So we have what is going on and why. I like this mechanism and I can see how to potentially make use of it later in my own workflows.

Til next time!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s