For the next generation of big data solutions, data must be made accessible to the masses to deliver on business needs at scale. By combining the power of AI and automation with big data analytics, enterprises can achieve new levels of speed and agility to their analytics initiatives without risking scale, security, and maintainability.” – says Sean Knapp, Founder and CEO of Ascend.io., in an exclusive interview with Enterprise Talk.
ET Bureau: With businesses becoming more data-driven, how can autonomous, intelligent platforms help them move ahead with optimal efficiency?
Sean Knapp: Increased competition and customer demands mean that data teams today are being asked to do more with data, and faster, than ever before. When you combine that with the fact that 97% of data teams are at or above capacity, it is no surprise these teams are increasingly turning to platforms that enable them to create more value for the business without the heavy lifting of traditional data pipelines.
By leveraging automation, data-aware intelligence, and flexible coding interfaces, data teams are freed from largely manual processes like maintenance and debugging lower-level systems, enabling them to focus more time on downstream projects.
And with highly automated systems, every business can enable more of its company to not only build faster on data but to do so safely with proper governance.
Read More: How the right IT vendors can help businesses
Finally, “flex code” solutions can provide the ease of use of low- and no-code interfaces with the full customizability of high-code solutions.
This puts more time back in data teams’ hands, so they can spend critical cycles on innovation and solving complex business challenges.
ET Bureau: How can the gaps in the modern data ecosystem impact businesses’ overall performance?
Sean Knapp: When it comes to the modern data ecosystem, there have been impressive technological innovations in business intelligence (BI), data processing, data warehousing, and other applications.
However, there is still a significant gap in connecting and orchestrating the movement and transformation of data between these business-critical systems.
Data pipelines have emerged as the standard technique for data movement and orchestration, but traditional pipelines are static, extremely brittle, and require significant data engineering resources.
According to a recent survey of data professionals, 54% of data engineers cited that their work was most impeded by maintaining existing and legacy systems, such as traditional pipelines.
Meanwhile, data scientists cited access to data and systems as their #1 bottleneck, again highlighting the pains felt by data engineering and those who depend upon them.
Ultimately, the impact is felt company-wide as teams struggle with slower, less reliable access to data, hamstringing innovation cycles, and impacting the business’s ability to produce actionable insights with data.
Organizations are quickly discovering that data engineering productivity is essential to unlocking data value and removing bottlenecks across the entire data team.
ET Bureau: Can applying a declarative approach to the data pipeline domain radically develop the level of automation?
Sean Knapp: Moving away from an imperative, task-based design to declarative, data-centric automation is the key to creating high-performing, automated pipelines that accelerate the time from prototype to production.
Declarative systems are able to deliver such incredible benefits as they are generally paired with domain-specific control planes, highly automated engines.
We’ve seen declarative systems gaining adoption up and down the technology stack, from Terraform to Kubernetes to React – each delivering on the promise of less code, faster development cycles, and significantly lower maintenance burden than their traditional, imperative predecessors.
ET Bureau: With the increasing demand for information and insights, data teams are asked to do more with less. How can Ascend.io assure the democratization of data engineering to offer real-time business needs?
Sean Knapp: Through advanced AI and automation, the Ascend Unified Data Engineering Platform revolutionizes data pipelines by freeing data teams from the monotony of traditional ETL, allowing teams to build pipelines that dynamically adapt to changes in data, code, and environment.
The study I cited earlier highlighted that only 3% of data teams have the capacity for new projects, which means the vast majority of teams need to find a better way to deliver data breakthroughs — not bottlenecks.
With the platform’s flex-code interface and declarative approach to orchestration, we democratize data engineering and bring it within reach to a broader group of “citizen data engineers.” Besides, Ascend enables teams to create data pipelines up to 10x faster and with 95% less code to meet the rapidly increasing data needs of businesses finally.
ET Bureau: Big data is fast becoming the key player in enterprise strategies. How do you combine the tools of AI and Automation with big data analytics to offer next-gen solutions? What role do Qubole and Ascend.io play in this?
Sean Knapp: For the next generation of big data solutions, data must be made accessible to the masses to deliver on business needs at scale.
By combining the power of AI and automation with big data analytics, enterprises can achieve new levels of speed and agility to their analytics initiatives without risking scale, security, and maintainability.
Ascend.io’s integration with Qubole achieves exactly that by combining the strengths of both platforms – bringing an advanced data pipeline automation technology with the most comprehensive data lake platform.
Our team automate the underlying engineering complexity of data pipelines to increase the productivity of data teams dramatically and vastly reduce the time required to extract valuable business insights from real-time, large-scale analytics.
Founder and CEO of Ascend.io. Prior to Ascend.io, Sean was a co-founder, CTO, and Chief Product Officer at Ooyala. Before founding Ooyala, Sean was the technical lead for Google’s legendary Web Search Interface team, leading initiatives that increased Google’s revenues by over $1B.