Load Data

SynxDB integrates with external systems, including big data systems, using external tables. This approach leverages a variety of loading tools and protocols, allowing you to choose the most suitable method based on the data source. The system loads external data by creating an external table that references the source, and then uses SQL commands to read data from or write data to it.

• Data Loading Overview
  • Data loading process
  • Loading tools and scenarios
• Load Data from Local Files
  • Load Data Using COPY
  • Load Data Using gpfdist
  • Load Data Using the file:// Protocol
  • Load Data Using gpload
• Load External Data Using Foreign Table
  • Use foreign table
  • Query a foreign table
  • Write data to a foreign table with ordered insertion
• Load Data from Web Services
  • Command-based external web tables
  • URL-based external web tables
• Load Data Using PXF
  • Install and configure PXF
  • Next steps
• Load Data from Kafka Using Kafka FDW
  • Basic usage
  • Supported data formats
  • Query
  • Message producer
  • Data import
• Load Data from Object Storage and HDFS
  • Install the extension
  • Instructions
  • HDFS and OSS-related configuration parameters
• Load Data from Hive Data Warehouse
  • Supported Hive file formats
  • Usage limitations
  • Steps
  • Examples of syncing tables
  • Data type mapping
  • Known issues
• Load Data from MySQL Server Using MySQL_FDW
  • Prerequisites
  • Use MySQL FDW
  • Usage Examples
• Connect to a Remote SynxDB
  • Use cases
  • How to use
  • Important notes
• Custom Multi-Character Delimiters for Reading and Writing External Tables
  • Compilation and installation
  • Usage example

• Load Data Using Apache NiFi
  • How it works
  • Key benefits of using NiFi with SynxDB
  • Example Use Case: Ingesting Log Files