“Once properly designed and implemented, data scientists, business analysts and other users have easy access to reliable and easy to access data to generate actionable business insights which can be further enhanced by the use of AI/ML tools,” says Kamal Maggon, Manufacturing and Consumer, Hexaware in an exclusive interview with EnterpriseTalk.
ET Bureau: Can you describe some of the critical digital transformation enterprises are facing in 2021? What steps can they take to mitigate them?
Kamal Maggon: Enterprises and IT departments are under tremendous pressure today to do more with fewer resources. It is critical for companies to unlock the power of data to make informed business decisions. For instance, a large mining company operating a fleet of vehicles with numerous sensors generates data volume in scale of petabytes.
The cloud is essential to store such enormous Volumes of data in the right Data Warehouse, manage the Velocity of trillions of messages per hour, and the Variety of data structures involved such as JSONs and XMLs.
The siloed nature, slowness and high costs of legacy systems have led executive teams across industries to adopt cloud-based analytics to solve issues on the ground in real time. The cloud makes it easy to continually optimize infrastructure requirements by scaling data warehouse volume and compute capacity separately.
It is easier to migrate to the cloud than ever before. The limitations of rehosting or refactoring on-premise data warehouses to the cloud have been overcome through automation driven migration. There are state-of-the-art tools today that deliver more than 60% reduction in TCO during cloud modernization, for instance, in shifting from Teradata to Snowflake.
ET Bureau: What steps can enterprises take to eliminate silos present in the cloud?
Kamal Maggon: One of the objectives of moving to Cloud is to eliminate silos, however at times, particularly in a direct lift and shift migrations, shadows of silos may appear in the cloud. Enterprises can address this by ensuring a smooth migration from their legacy on-premise systems.
Staying true to the objectives of cloud migration like scalability, interoperability and performance and incorporating these during design helps mitigate these risks. What works well is a structured approach to move workloads from on-premise to cloud – such as in the earlier mentioned example of migration from Teradata to Snowflake.
It is important to begin with an assessment of the customer’s unique needs as well as cloud costs, security, and architecture. A governance model should be put in place to guide the transformation, before finalizing the automation approach. This will ensure that the voice of the customer does not get diluted during the various stages of cloud adoption.
ET Bureau: How can IT teams effectively manage, store, and analyze a significant volume of data on on-premises data warehouses (DW)?
Kamal Maggon: Today’s problems can’t be solved with old technology. Storing, managing, and analyzing huge volumes of data requires a move to the cloud, as in the case of industries such as mining and manufacturing. Moving to cloud helps split the stages of data such as ingestion, preparation (from cleansing to ready to use) and analysis. This improves efficiency as heavy transformation speed and capacity are available on demand.
The data landscape for each client must be reviewed for overall maturity, and readiness to meet strategic objectives. Data volumes, diversity of sources, security and governance are all important considerations in designing the right data ecosystem.
Once properly designed and implemented, data scientists, business analysts and other users can have easy and reliable access to data, enabling them to generate actionable business insights which can be further enhanced by use of AI/ML tools.
ET Bureau: What trends can enterprises expect that will transform data warehouse management in the foreseeable future?
Kamal Maggon: Data Warehouses have evolved over the years from an integrated data store to an analytical querying platform. Today, data warehouses are compute engines that transform data and also serve as data platforms that democratize data access. In the near future, data warehouses will evolve into ML engines with the capability to govern stored data with business glossary and data quality metrics.
These will result in the increased adoption of integrated AI-ML driven data platforms that will increase the effectiveness of business insights while optimizing IT costs for the enterprise.