INDUSTRY: ENERGY
Origin Energy
About Origin Energy
Origin Energy is at the forefront of Australia’s energy market, serving more than 4 million customers. The company relies on a range of energy sources for power generation, from traditional fuels such as coal and gas to renewables such as wind and sun. It analyzes data including consumption patterns and environmental trends to understand and better serve its customers.
Although an established player, Origin is facing headwinds in terms of new, nimble competition. Smart meters and personal-use solar photovoltaic panels have created a distributed energy market that creates challenges for a traditional energy provider to forecast customer demand accurately.
Origin needed to adopt an intensive customer-centric approach using big data analytics that would allow it to connect with customers and monitor energy usage easily. This would spur the creation of innovative schemes to recruit new customers while retaining existing ones, as well as helping Origin to forecast demand.
The Challenge: traditional infrastructure unable to keep up with business growth
Origin needed a data management solution that could break down huge silos of data from various sources, merge the data and prepare it for analytics fast. Until 2016, Origin used on-premises hardware to run all its operations and big data projects. However, with data growing exponentially, it needed to rethink infrastructure. As the volume and complexity of data increased in line with growth, the existing infrastructure was struggling to support the increasing data demands.
James Moor, General Manager of Data and Analytics at Origin Energy says, “We just couldn’t crunch the numbers fast enough on premises and needed an infrastructure configuration that would support what we needed to do”.
Another issue was data from SAP systems and other legacy sources such as Oracle and SQL server sources were difficult to extract. Slow-performing queries and cumbersome firewalls had to be overcome in order to access and prepare the data before Origin’s analysts could even get access for data mining and analytics.
The client faced several issues:
- Existing infrastructure and data access could not handle the rapidly increasing analytical demand
- Fragmented storage slowed access to critical data, stifling innovation.
- Cost of physical infrastructure & FTE overload.
- Access to SAP data and integrating it with other enterprise data.
- Multiple data silos meant there were multiple versions of the “truth” and more time was spent wrangling data rather than analysing it.
The Solution: Shifting to the Cloud with AWS and BryteFlow
Moving to the cloud an enterprise analytics environment with centralized data access was the only viable option to support Origin’s data initiatives.
- BryteFlow software was used to create an automated data lake for Origin based on three core AWS products: Amazon Redshift, Amazon Simple Storage Service (Amazon S3), and Amazon Elastic MapReduce (Amazon EMR) and allowed easy integration with AWS services like Amazon SageMaker for machine learning. The data available for reporting and analytics could be used with very little effort for machine learning, hence future proofing the data.
- The BryteFlow software continually replicated and synced Origin’s on-premises data to a data lake created on Amazon S3 and to a data warehouse on Amazon Redshift with minimal impact on sources.
- BryteFlow handled the large amount and complex enterprise data with ease and is now an enterprise grade, resilient solution.
- BryteFlow (configured for Amazon EMR) enabled data transformations on extracted raw data from different sources and prepared it for mining.
The Result: fast access to data and insights lead to innovation and high levels of customer satisfaction
Origin’s new enterprise analytics environment has been a game-changer for the company’s data operations and has yielded several major benefits:
- BryteFlow software has helped to achieve accurate data replication on the AWS Cloud with low latency, facilitating faster time-to-market for new and highly personalized customer offerings.
- Significant reduction in data costs.
- Company data is now accessible to functional teams across the organization by consolidating all workloads and databases into one powerful engine, accommodating increasing volumes of data and users.
- SAP data can easily be integrated with other enterprise data sets.
- Data access has improved from several days to mere hours.
- It has enabled fast and efficient response to consumer requests.
- Scalability and flexibility for new use cases for the data is helping in adopting new data-led initiatives.
