SAP Data Replication

The SAP Cloud Connector and Why It’s So Amazing

In this blog we look at the fascinating world of SAP Connectors, their different types – with a special focus on the SAP Cloud Connector. We examine how the SAP Cloud Connector works, its benefits and uses and how BryteFlow can be used as an automated SAP Cloud Connector to provide an easy, secure, no-code path from SAP systems to Cloud platforms. About BryteFlow SAP Data Lake Builder

Quick links

About SAP

SAP is one of the most widely used ERP software for modern organizations around the world. SAP systems are key to centralized data management and for adding efficiency and automation to business processes. SAP ERP is modular, and organizations can choose from a wide range of modules, to pick the ones most relevant to their business. Earlier different departments of an organization would have data stored in separate databases, for e.g. inventory, sales, suppliers etc. and this data would not be easily accessible to users in other departments. SAP allows organizations to centralize the data, accelerate workflows, and to use it for business analytics to increase productivity. BryteFlow for SAP DB Replication
SAP solutions have evolved with the times and today they can be installed on-premise, on the Cloud or even as a hybrid implementation. SAP offers more than 100 products customized for various business applications. In addition, there is an ecosystem of SAP partner vendors and consultants who offer more products and services to customers. As you can see, SAP systems generate a lot of data, and it needs to be extracted and delivered from SAP internal and external systems to Cloud data warehouses and repositories for consumption. SAP connectors play a big role in this. About BryteFlow SAP Data Lake Builder 

What are SAP Connectors?

SAP connectors provide the means for SAP systems to integrate with different applications and technologies by use of open standards. SAP connectors enable technical operations between SAP components, usually coded as ABAP or ABAP objects, with components typically coded in C++, Java, .NET, Visual Basic etc. SAP HANA to Snowflake (2 Easy Ways)
App Connect and the SAP instance can have bi-directional communication using the SAP connector. The connector supports BAPI (Business Application Programming Interface), RFC (Remote Function Call), and IDOC (Intermediate Document) interfaces. How to Carry Out a Successful SAP Cloud Migration

Types of SAP Connectors

SAP has 6 basic connectors that can enable exchange of data between various external systems and tech environments and SAP ABAP systems. RFC, HTTP and IDocs are the underlying supported technologies for two-way communication. SAP to Snowflake – Easy Integration

SAP connectors function in 2 ways:

Client Case Functionality
The external system executes functionality by sending inputs into the ABAP system and receives the results of the operation in the response. RISE with SAP (Everything You Need to Know)

Server Case Functionality
The ABAP system executes functionality by sending inputs to the external system and receives the results of the external operation in the response. SAP HANA to Snowflake (2 Easy Ways)

Here are some SAP Connectors we will touch upon.

SAP NetWeaver RFC Software Development Kit (NW RFC SDK)
If it is possible to extend the external system with C/C++ coding, the NW RFC SDK is an ideal connector. It also provides bindings for other programming languages, with a native C interface, like Perl, Python, PHP or Go. It supports communication via RFC and WebSocket RFC. SAP BW and how to create an SAP OData Service

SAP Java Connector (JCo)
If the external system can be extended via Java Coding, JCo can be used for connecting to an SAP ABAP system. The connector supports communication technologies such as RFC, WebSocket RFC and IDoc.  SAP ECC and Data Extraction from an LO Data Source

SAP Connector for Microsoft.NET (NCo)
Also called SAP NET connector, this connector’s function is to integrate Microsoft .NET-based environments or Windows front-end applications with an SAP ABAP system. It supports RFC and WebSocket RFC communication. No Need to Panic with SAP’s Ban of RFC

SAP Business Connector (SAP BC)
SAP Business Connector is a middleware server unlike the previously mentioned connectors which are SDKs (software development kits). The development kit connectors allow integration using scripting at the source code level while the SAP Business Connector integrates external systems whose source code cannot be modified or is not available. The external systems have a public interface for communication (e.g., XML, JSON, OData, SOAP, EDI, or flat file) and use standard network protocols like HTTP(S), FTP, Email, MIME, or JDBC. It supports RFC, WebSocket RFC, IDoc, HTTP, FTP, SMTP, IMAP, POP3, and JDBC as communication technologies. CDS Views in SAP HANA and how to create one

In addition, SAP Business Connector provides:

  • Mapping tools to transform data formats of external systems like some XML trees, JSON components, EDI records etc. into a data format recognized BY SAP systems such as IDOC segments or RFC tables. Reliable SAP CDC and Data Provisioning in SAP
  • A general-purpose XML parser and renderer.
  • An HTML parser for extracting data from HTML pages e.g. online inventory systems or catalogues
  • An XSLT mapping engine.
  • Tools for composing/decomposing multi-part MIME/SMIME messages.
  • Tools for encrypting/decrypting or digitally signing messages.
  • Tools for data operations such as zip/gzip, Base64 encoding, and URL encoding.
  • A Transaction Store for logging and monitoring transactional messages.
  • A Job Scheduler for executing random periodic jobs.
  • It also allows you to extend functionality by adding custom modules written in Java or C/C++ if predefined tools do not meet requirements.
  • Supported communication technologies are RFC, Websocket RFC, IDoc, HTTP, FTP, SMTP, IMAP, POP3 and JDBC.

SAP Open Connectors
SAP Open Connectors are a part of the SAP Integration Suite and are meant to accelerate and standardize connections with third-party applications and systems. They are a set of over 160 pre-built connectors that can connect without needing any custom coding. SAP Open Connectors provide open RESTful APIs and support JSON data format services. SAP BODS (What makes it a great SAP ETL Tool)

With SAP Open connectors you can avoid manual data entry, automate important business processes and minimize errors. They enable data consistency, seamless operations without coding, and can even be transported to a different environment in JSON format. SAP Open connectors enable connections to different applications, data sources and services through APIs (Application Programming Interfaces) and are very user-friendly, making it simple for SAP customers to configure and manage integrations.  SAP SLT in HANA: How to use SLT Transformation Rule

SAP Cloud Connector (SCC)

Now that we have discussed types of SAP connectors in general, let’s move on to the hero of our story, the SAP Cloud Connector. The SAP Cloud Connector provides a safe way for your on-premise internal SAP systems to interact with SAP applications on SAP BTP (Business Technology Platform), which is a Cloud-based platform. The Cloud Connector provides a safe, secure TLS (Transport Layer Security) tunnel between an on-premise SAP system and a Cloud-based application on SAP BTP. Other BTP applications that can use SAP Cloud Connector to connect include SAP S/4HANA Cloud, SAP IBP, and SAP Data Intelligence. Cloud applications do not need to open a port on the corporate firewall for incoming traffic but can connect using SAP backend functionality. SAP S/4 HANA and 5 Ways to Extract S/4 ERP Data

The SAP Cloud Connector is highly secure
The SAP Cloud Connector operates on an ‘explicit allow’ principle for security, requiring explicit permission for all services (RFC, OData, etc.) to ensure secure communication, giving you granular control over access to your on-premise applications. You can control which backend systems can be accessed from the Cloud through Access Control Lists and which resources can be executed, including HTTP service paths and RFC-enabled function modules. Communication technologies supported are RFC, HTTP, JDBC/ODBC, LDAP, and general-purpose TCP. SAP prohibits RFC use for extraction

The SAP Cloud Connector has versatile connectivity, it can be used in multiple scenarios

  • It can be used for connecting SAP or non-SAP applications.
  • SAP Cloud Connector can connect a single Cloud system to a single on-premise system.
  • It can connect multiple Cloud systems to multiple on-premise systems.
  • One Cloud Connector can connect to multiple SAP BTP subaccounts.
  • Multiple Cloud Connectors can connect to one or more SAP BTP subaccounts.

The SAP Cloud Connector can be installed on multiple platforms
The SAP Cloud Connector can be installed on Windows, Linux, or Mac OSX, either on a virtual or physical machine. It requires an SAP BTP account to configure the connection between the SAP Cloud environment and the Cloud Connector instance. Multiple Cloud Connectors can be deployed based on your landscape and objectives, for connecting to one or more SAP BTP region-subaccounts. How to set up SAP Cloud Connector 

SAP Cloud Connector Features and Benefits

  • SAP Cloud Connector provides a secure path between on-premise systems and SAP BTP applications.
  • SAP Cloud Connector is easy to set up with a clear configuration of the systems connected to SAP Cloud Platform. SAP to AWS (Specifically S3) – Know an Easy Method
  • By using the SAP Cloud Connector you avoid exposing your whole internal IT environment to risk, instead it enables connection only to specific on-premise assets. Reliable SAP CDC and Data Provisioning in SAP
  • SAP Cloud Connector functions as an on-premise agent within a secure network.
  • SAP Cloud Connector recovers automatically from broken connections. How to create an SAP OData Service on SAP BW
  • SAP Cloud Connector functions as a reverse invoke proxy between the on-premise network and SAP BTP, so you don’t need to configure an on-premise firewall to enable external access from the Cloud to internal systems. SAP HANA to Snowflake (2 Easy Ways)
  • SAP Cloud Connector provides granular access control over on-premise systems and resources, and Cloud applications using the connector. SAP Extraction using ODP and SAP OData Services
  • A single SAP Cloud Connector can connect with multiple backend SAP systems. How to Carry Out a Successful SAP Cloud Migration
  • SAP Cloud Connector enables use of features needed for vital business scenarios.
  • SAP Cloud Connector performs audit logging of inbound traffic and changes in configuration.
  • The SAP Cloud Connector can run in a high-availability setup. To avoid possible disruptions, the SAP Cloud Connector should be configured for high availability with a failover machine ready to maintain connectivity if the primary instance fails. SAP HANA CDC to Snowflake with BryteFlow

SAP Cloud Connector Use Cases

The SAP Cloud Connector allows specific applications to connect to and access other specific applications, without exposing the applications to general access. Here are some use case scenarios. Reliable SAP CDC and Data Provisioning in SAP

SAP Cloud Connector helps integrate on-premise systems with Cloud applications

  • Using Cloud Integration, which is included in the SAP Integration Suite, organizations can integrate on-premise systems with Cloud applications via many different protocols including LDAP, HTTP, RFC, OData, SFTP etc. SAP Extraction using ODP and SAP OData Services
  • Using SAP API Management, another component of the SAP Integration Suite, enables customers to use SAP Cloud Connector to expose internal APIs to the Internet so external systems and partners can communicate with them using a secure API gateway. Create a CDS View in SAP HANA

SAP Cloud Connector allows customization of system extensions

  • Customers can use the SAP Cloud Connector functionality to build and host custom applications in SAP BTP, for easier communication with on-premise systems and to extract the required data, if the needed functionality is lacking. For instance, you might use SAP Graph to retrieve data from the on-premises SAP S/4HANA. The custom application, hosted in SAP BTP, can communicate with the on-premises system to fetch the required data. Thus, organizations have the appropriate functionality on hand when needed. SAP SLT in HANA: How to use SLT Transformation Rule
  • SAP Ariba Cloud Integration Gateway on SAP Ariba, also uses SAP Cloud Connector to connect with internal systems, enhance communication between internal and external applications, and boost functionality of SAP BTP services, both inside and outside of SAP Cloud. Various SAP BTP services, such as SAP Asset Manager, SAP Data Warehouse Cloud, and SAP Cloud Integration, use the Cloud Connector to connect with internal systems. What is SAP ECC and Data Extraction from an LO Data Source

SAP Cloud Connector enables advanced data processing

  • With the SAP Cloud Connector, organizations can extract and transfer large volumes of data for analytics to SAP systems like SAP Data Warehouse Cloud and SAP HANA Cloud to process data and analyze it. SAP BODS (What makes it a great SAP ETL Tool)
  • Customers can use the SAP Cloud Connector to replicate and virtualize data from an on-premise SAP HANA system to SAP HANA Cloud, so valuable business data is accessible in a Cloud environment for faster analytics and insights. How to Migrate Data from SAP Oracle to HANA
  • SAP Analytics Cloud users can use Cloud Connector to securely access data from on-premise/private cloud systems like SAP HANA, SAP S/4HANA, or SAP BW/4HANA. It can also connect to OData APIs exposed by internal systems to create dashboards and reports based on your data. SAP to Snowflake (Make the Integration Easy)

SAP Cloud Connector Advantages

SAP Cloud Connector enables seamless integration of On-premise and Cloud systems
Using the SAP Cloud Connector makes life easier when you have on-premise SAP systems and some SAP applications in the Cloud. The Cloud Connector seamlessly integrates the two, enabling businesses with on-premise systems to interact with Cloud-based services, so valuable, timely data can be easily accessed, exchanged and consumed. How Rise with SAP makes Cloud Migration easy   

SAP Cloud Connector provides high grade security
The SAP Cloud Connector enables a highly secure tunnel between on-premise systems and Cloud applications, ensuring sensitive data remains protected from unlawful access. It uses standard encryption and authentication measures to ensure integrity of data. It provides convenience since you do not need to open a port on the company firewall for incoming traffic, but connection can be made using SAP backend functionality. How to have a successful SAP Cloud Migration

SAP Cloud Connector enables Real-time Connectivity
Using SAP Cloud Connector, companies can get real-time connectivity, and immediate access to Cloud applications and data. This feature is a must-have for organizations that rely on real-time data for operational efficiency and for taking informed decisions. Real-Time SAP Replication with BryteFlow

SAP Cloud Connector has high scalability
Increasing data volumes, the addition of new applications and services and enterprise growth mean that your SAP Cloud Connector needs to scale up in tandem to handle the increased demand. Fortunately, the SAP Cloud Connector is equal to the task and can scale easily with growing volumes, keeping the integration’s efficiency intact. About BryteFlow SAP Data Lake Builder

SAP Cloud Connector is compatible with the SAP Cloud Platform
The SAP Cloud Connector was created for the SAP Cloud Platform and is perfect for organizations using SAP systems. It allows SAP customers to easily integrate and effectively use the complete lineup of SAP applications and services available on the SAP Cloud Platform. SAP BODS, a great ETL Tool

BryteFlow as a Third-Party SAP Connector

Besides the SAP Cloud Connector, you could also consider third-party tools that act as SAP connectors to connect and transfer data securely from on-premise SAP systems to the Cloud or vice versa. These connectors include well-known names like Fivetran, Qlik Replicate, Informatica and our own BryteFlow. SAP S/4 HANA and 5 Ways to Extract S/4 ERP Data

BryteFlow as an SAP Connector
BryteFlow SAP Data Lake Builder can be used to transfer data from SAP On-Premise and Cloud applications to Cloud hyperscalers like AWS, Azure and GCP, to destinations such as Snowflake, Redshift, S3, Azure Synapse, ADLS Gen2, Azure SQL Database, Google BigQuery, Postgres, Databricks, SingleStore and SQL Server. Meanwhile BryteFlow Ingest provides ETL of data from SAP databases in real-time to on-premise and Cloud destinations, provided access to the underlying database is available. BryteFlow for SAP CDC to Snowflake

BryteFlow for SAP Migrations
In certain cases, BryteFlow can also transfer data in real-time to SAP HANA destinations. For e.g. migration from SAP ECC Release 3 and 4, SAP database on Oracle, SAP database on SQL Server, SAP S/4HANA Cloud Private Edition (new SAP Rise Cloud- Runtime version), SAP on Oracle to SAP HANA DB. How to Migrate Data from SAP Oracle to HANA

BryteFlow SAP Data Lake Builder provides very high throughput and low latency
Our BryteFlow SAP connector loads data with very high throughput, transferring 1,000,000 rows of data in 30 seconds approx. It is also fast to deploy, and you can start receiving data in just 2 weeks. How to Carry Out a Successful SAP Cloud Migration

BryteFlow uses SAP-certified data extraction methods for ETL
You should note that BryteFlow is one of the very few SAP connector tools that employs the data extraction method recommended by SAP, using ODP and OData Services to extract SAP data in comparison to other tools that use Remote Function Calls (RFCs). This is important since according to SAP Note 3255746, SAP will not permit the use of RFCs for data extraction, except for SAP-internal applications. Read about the SAP ban of RFCs

BryteFlow has flexible connections for SAP
The BryteFlow SAP Data Lake Builder has multiple flexible connections for SAP including SAP BW Extractors, Database Logs, SAP CDS Views, S/4HANA, ECC, SAP SLT and SAP Data Services. etc. It can extract the data and load it to any Cloud platform including AWS, Azure or GCP in real-time. SAP to AWS S3 – Know an Easy Method

Our SAP connector tool supports loading of high volumes of enterprise data
The BryteFlow SAP connector supports the initial full refresh of petabytes of enterprise data, loading huge volumes rapidly using parallel, multi-thread loading, smart partitioning and compression, and brings across incremental data and deltas in real-time using Change Data Capture. The BryteFlow connector merges the deltas (Updates, Inserts, Deletes) automatically with initial data to sync data with the SAP source in real-time. SAP HANA to Snowflake (2 Easy Ways)

Our SAP Connector extracts data with business logic intact
Our SAP connector can extract data from SAP applications like SAP ECC, S4HANA, SAP BW, SAP HANA and replicate data with business logic intact to the target. You get a completely automated setup of data extraction and automated analysis of the SAP source application. CDS Views in SAP HANA and how to create one

BryteFlow SAP Data Lake Builder automates SAP ETL processes
The BryteFlow SAP connector automates every process including SAP data extraction, Change Data Capture, merging, masking, schema and table creation, DDL and SCD Type 2 history – no coding required. SAP to Snowflake (Make the Integration Easy)

BryteFlow connector delivers data that is ready to use on target
BryteFlow performs data type conversions out-of-the-box (e.g. Parquet-snappy, ORC). Thus, data delivered can be immediately used for analytics with BI tools of choice or for Machine Learning Models. Our SAP connector delivers data to Snowflake, Databricks, Redshift, S3, Azure Synapse, ADLS Gen2, Azure SQL DB, SQL Server, Kafka, Postgres and Google BigQuery.

Besides ETL from SAP applications, BryteFlow uses real-time CDC replication to deliver data from SAP databases
If access to the underlying database is available, the BryteFlow SAP connector can provide SAP replication with real-time log-based Change Data Capture from SAP databases. BryteFlow for SAP

Transferring data from SAP Oracle to SAP HANA DB with BryteFlow

For illustrative purposes, here is a step-by-step process to demonstrate how to move data from an SAP Oracle database to an SAP HANA database using BryteFlow Ingest. BryteFlow Ingest replicates data from SAP databases (provided you have access to the underlying database) while BryteFlow SAP Data Lake Builder delivers data from SAP applications. In either ase, you can move the data in just 5-6 steps.

Step 1: Download and Install BryteFlow Ingest. Login and Configure source database.
You have two options for obtaining the software: you can contact the BryteFlow team or download it from the AWS Marketplace or from the Azure Marketplace. Once it’s installed, log in and select the “Connections” link on the left panel. Select the ‘Source’ tab in the top nav bar. From there, follow these steps to set up your source database:

  • Choose “SAP (Oracle)” as the Database Type.
  • Enter the OracleHost Details in the Database Host field.
  • Provide the Port Details in the Database Port field.
  • Input the Database Details into the Database Name field.
  • Enter the User Id used for Database connection in the User Id field.
  • Input the corresponding User password for Database connection into the Password field.

Step 2: Configure destination database.
To establish a connection with your destination database, on the “Connections” screen select the “Destination” tab on the top nav. Follow these steps to set up the Destination Database.

  • From the dropdown menu, choose “SAP HANA JDBC” as the Database Type.
  • Specify the destination drive details within the “dst-drv” section.
  • Input your destination URL into the “dst-url” field.
  • Enter your user ID into the “User ID” field.
  • Provide the corresponding user password for the database connection in the Password field. Please note that certain fields may differ depending on the source database selected.

Step 3: Select the tables you want to replicate.
Select the “Data” link in the left panel to set up the source table(s) for replication. Here, you’ll define the primary key and transfer method for each table. Opting for ‘By Primary key with History’ automatically maintains SCD Type2 history. Alternatively, selecting ‘By Primary Key’ creates a mirrored copy of the source table without historical records. Additionally, you can use the “Where” feature to filter out undesired data. Byteflow Ingest offers the capability to “mask” sensitive information within columns.

Step 4: Schedule the replication.
Select the “Schedule” link in the left panel to visit the Schedule page. Here you can schedule the data transfer. You can opt for either “Automatic” or “Periodic” replication of deltas. “Automatic will deliver deltas as soon as there are changes at source while “Periodic” option enables you to create your own schedule, whether daily or weekly.
On the right you have four buttons at the bottom of the screen. Select “Full Extract” for starting the initial load. Select “Sync New Tables” to incorporate new tables after replication activation. Select “Sync Struct” to reflect structural changes in existing tables. Select “Rollback” to return to a previously successful operation, especially useful in case of replication interruptions.

Step 5: Monitor the transfer of data.
Select the “Dashboard” link in the left panel to get insights into ongoing data transfers, such as the overall count of loaded records and tables, live updates on ongoing loading operations, replication intervals, and the current state of connections.

Note: As an alternative tracking option, select the “Log” link in the left panel to track data loading progress via the log, and get instant notifications about the data replication. This covers extraction or loading statuses, the volume of loaded records, and any errors encountered within the process.

Conclusion
In this blog we learned about SAP Connectors of different types with a special emphasis on the SAP Cloud Connector, its benefits and use cases. We also saw how BryteFlow can be used as a completely no-code SAP connector, and it’s point-and-click journey to deliver data from SAP on Oracle to SAP HANA DB. Like to know more? Contact us for a Demo