Cloud computing is rapidly becoming the business standard for organizations who want to make use of its cost efficiency, scalability, and mobility. Most of our clients today now operate in a hybrid cloud environment that combines on-premises and cloud infrastructures, with orchestration between the two platforms. In response to this new standard, Azure Data Factory (ADF) was the first-generation solution for integrating disparate data sources that organizations may have spread throughout both on-premises and the cloud.

While many developers can’t live without the Data Movement feature in ADF, which orchestrates and automates the movement and transformation of data, that’s not to say the road to production in the cloud with Azure Data Factory has always been a smooth one.

What Limitations Does Azure Data Factory V2 Solve?

Anyone who uses Azure Data Factory knows that it can be tedious, time-consuming, and sometimes extremely frustrating to execute certain tasks.

The need for Azure Data Factory V2 (ADF V2) was made clear by the first version’s previous limitations. For instance, ADF was an optimal choice for data-based tumbling windows for loading data, but it wasn’t a great option for other incremental load methods, such as modified date and change tracking. ADF also had poor support for source control and automated deployments. These shortcomings led to challenges when promoting complex solutions to development, testing, and production. Perhaps one of the most frustrating limitations found in ADF’s first version was that it had absolutely no internal debugging. As you can imagine, that left the door open for many, many issues.

Luckily, the road to implementation has become a whole lot smoother with the release of ADF V2 – a seamless solution that lets current ADF users get past limitations without a huge rollout.

Here’s What ADF V2 Does Differently

You can view a list of new features here, but there are a couple new updates that deserve a special callout. I believe the following upgrades will be transformative for those who are all too familiar with some of the pain points listed above.

Lift and Shift SSIS:

With little or no modifications, you can now run your on-premises SSIS solution in the cloud without needing to deploy and configure a VM. If you want to create cloud-ready or cloud-first solutions faster, cheaper, and even more reliably than before, then ADF V2 is going to impress you!

Since many organizations have deep SSIS experience, the ability to run those packages in the cloud may also allow them to leverage existing skillsets and adopt the cloud faster.

Iterative Pipeline Development

Debugging in ADF v1 was non-existent, resulting in longer development times and more errors. The inclusion of real-time debugging features makes developing in ADF V2 more productive and significantly less frustrating.

A Powerful Solution at Your Fingertips

When we look back at ADF’s first version, we see a system that was off to a great start. It had good functionality yet lacked some of the features that developers today simply can’t move forward without. ADF V2 solves the issues left over from the first version, delivering a simple-to-implement, powerful platform.

If you’ve already been using the first version of ADF, then the changeover will be relatively simple. Even if you haven’t used the platform at all, BDO Digital can help set the course for a smooth transition. Contact us to learn more about how to leverage the latest technology to improve efficiencies and make your ongoing transition to the cloud a smooth one.