Mastering Oracle GoldenGate OBEY Files: Building Modular and Maintainable Data Synchronization Configurations

Late one night, you might receive an urgent production change request: add 10 new business tables to each of 5 running Extract processes. You take a deep breath, open the server terminal, and run EDIT PARAMS for each one. You carefully find the end of the TABLE statements in each parameter file, which are already hundreds of lines long, and then precisely copy and paste 10 lines of TABLE schema.table; configuration. Throughout the process, you must be fully concentrated, fearing a typo, a missing semicolon, or making a change on the wrong process, which could bring down the entire data synchronization link.

This scenario is a microcosm of the daily work of countless OGG administrators. As the number of tables and the complexity of business logic grow, our once-concise .prm parameter files inevitably evolve into a “monolithic application” that is difficult to maintain and fraught with high risk.

This article will solve this core pain point. We will delve into an elegant and powerful feature in Oracle GoldenGate—the OBEY file. After reading this article, you will master a professional methodology for refactoring chaotic configurations into a modular, reusable, and easily maintainable system, freeing you from the anxiety and inefficiency of the scenarios described above.

The Root of Chaos: The Drawbacks of Monolithic Parameter Files

Before diving into the solution, we must clearly understand the problem. An unoptimized, massive .prm file typically has several fatal flaws:

Extremely Poor Readability: Database connection information, DDL handling options, performance parameters, error handling logic, and hundreds of TABLE or MAP statements are all jumbled together, making it like reading an ancient script for newcomers.
High Maintenance Costs: As in the opening scenario, a simple request like “add a table” or “modify the global error handling strategy” turns into a nightmare of making repetitive changes across multiple files. This violates the most fundamental principle of software engineering: DRY (Don’t Repeat Yourself).
Prone to Errors: Manually synchronizing changes across multiple files is a classic “fat finger error” zone. Configuration inconsistencies are often the root cause of “mysterious” failures in a sync link.
Zero Reusability: When you need to create a new Replicat process that is highly similar to an existing one, you have no choice but to copy the entire file and then carefully modify it. The configuration cannot be reused as an independent “component.”

These problems might be tolerable in a small-scale environment with a limited number of tables. But in a complex, enterprise-level environment, they are potential “time bombs.”

Getting Started with OBEY Files: Basic Syntax and Principles

The OBEY file is not a sophisticated technology; it is more of an embodiment of the “convention over configuration” design philosophy within OGG.

Definition: An OBEY file (usually with an .oby suffix, though this is not mandatory) is a plain text file containing OGG parameters. Its sole mission is to be called by a main parameter file (.prm) via the OBEY command.

How It Works: When an OGG process (like Extract or Replicat) starts and parses its .prm file, as soon as it encounters an OBEY command, it pauses parsing the current file. It then opens and fully executes all parameters within the called OBEY file. Once the OBEY file is finished, the process returns to the main file and continues parsing from the line following the OBEY command. This process is analogous to #include in C or source in a Shell script.

Syntax:

OBEY <file_path/file_name>

File Path: You can use an absolute path, but the best practice is to use a relative path. OGG uses its startup directory as the base. By default, all our parameter files and OBEY files should be stored in the dirprm subdirectory of the OGG installation directory. This makes the configuration highly portable.

A Simple Example: Let’s say we want all Replicat processes to use a unified set of database session settings.

In the dirprm directory, create a file named common_settings.oby:

-- common_settings.oby
-- This file contains standard settings for all Replicat processes.
DBOPTIONS SUPPRESSTRIGGERS
DBOPTIONS DEFERREFCONST

Call it from your Replicat parameter file, rep1.prm:

-- rep1.prm
REPLICAT rep1
USERIDALIAS ogg_tgt_alias DOMAIN OracleGoldenGate

-- Include common settings
OBEY common_settings.oby

-- Specific MAP statements for this process
MAP src.customers, TARGET tgt.customers;
MAP src.orders, TARGET tgt.orders;

Just like that, when the rep1 process starts, it will automatically apply the DBOPTIONS parameters without you having to write them repeatedly in the main file. If you need to add a new common parameter for all processes in the future, you only need to modify the common_settings.oby file.

OBEY Files in Action: Four Classic Application Scenarios

The theory is simple, but the power of OBEY files is demonstrated in practical application scenarios. Here are four of the most widely used patterns in enterprise environments.

Scenario 1: Centralized Table List Management (`TABLE`/`MAP`)

This is the most basic yet most effective use of OBEY files.

Pain Point: Multiple Extract processes (e.g., a real-time Extract and a batch Extract for a downstream data warehouse) need to capture the same set of core business tables.

Solution:

Create a core_tables.oby file specifically to store the list of these tables.

-- core_tables.oby
-- List of core business tables for extraction.
TABLE fin.gl_ledgers;
TABLE fin.gl_je_headers;
TABLE fin.gl_je_lines;
TABLE ar.ra_customer_trx_all;
TABLE ar.ra_customer_trx_lines_all;

In your real-time Extract (extfin.prm) and batch Extract (extdw.prm), replace the lengthy list with a single OBEY line.

Before (extfin.prm):

EXTRACT extfin
USERIDALIAS ogg_src_alias DOMAIN OracleGoldenGate
EXTTRAIL ./dirdat/fn
-- Long list of tables
TABLE fin.gl_ledgers;
TABLE fin.gl_je_headers;
TABLE fin.gl_je_lines;
TABLE ar.ra_customer_trx_all;
TABLE ar.ra_customer_trx_lines_all;
-- ... other parameters

After (extfin.prm):

EXTRACT extfin
USERIDALIAS ogg_src_alias DOMAIN OracleGoldenGate
EXTTRAIL ./dirdat/fn
    
-- Include the list of core finance tables
OBEY core_tables.oby
    
-- ... other parameters

Benefit: When the business needs to add a new core table (e.g., fin.ap_invoices_all), you only need to add one line to the core_tables.oby file. All processes that depend on this file will automatically pick up the change after a restart, significantly reducing the cost and risk of changes.

Scenario 2: Creating a Standardized Configuration Library

An enterprise-level OGG environment must have uniform standards. OBEY files are the perfect tool for enforcing them.

Pain Point: How can you ensure all Replicat processes adhere to the company’s standard error handling, DDL synchronization, and conflict resolution strategies?

Solution:

Create a “Standard Replicat Configuration Library” file, for example, std_replicat_config.oby.

-- std_replicat_config.oby
-- Standard configuration library for all Replicat processes.
-- Enforces company-wide policies.

-- DDL Handling
DDL INCLUDE MAPPED

-- Error Handling: Log errors for later analysis, but do not abend the process.
REPERROR (DEFAULT, DISCARD)

-- Collision Handling: Overwrite if a record exists on insert.
HANDLECOLLISIONS

Require all new Replicat parameter files to include this OBEY file at the beginning.

Benefit: This approach transforms configuration best practices from “a document” into “a piece of executable code,” enforcing consistency and stability across all sync processes. Auditing and handovers also become incredibly simple.

Scenario 3: Modularizing Complex Logic (e.g., `COLMAP`)

For heterogeneous replication or scenarios requiring data cleansing and transformation, the COLMAP clause within the MAP statement can become extremely bloated.

Pain Point: A COLMAP with dozens of field mappings and transformation functions can make the main parameter file’s readability plummet.

Solution:

Separate the complex COLMAP logic into an independent OBEY file. Note that OBEY can be used inside a MAP statement.

Before (repsales.prm):

REPLICAT repsales
...
MAP sales.orders, TARGET bi.f_orders,
COLMAP (USEDEFAULTS,
    order_id = order_id,
    order_date = @DATE('YYYY-MM-DD', 'YYYY/MM/DD', order_date),
    customer_id = customer_id,
    order_total = order_value * 1.1, -- Add tax
    order_status = @CASE(status, 'P', 'Pending', 'S', 'Shipped', 'C', 'Cancelled', 'Unknown'),
    -- ... dozens more lines ...
    source_system = 'OLTP'
);

After: First, create the map_orders_colmap.oby file:

-- map_orders_colmap.oby
-- Column mapping logic for the sales.orders table.
COLMAP (USEDEFAULTS,
    order_id = order_id,
    order_date = @DATE('YYYY-MM-DD', 'YYYY/MM/DD', order_date),
    customer_id = customer_id,
    order_total = order_value * 1.1, -- Add tax
    order_status = @CASE(status, 'P', 'Pending', 'S', 'Shipped', 'C', 'Cancelled', 'Unknown'),
    -- ... dozens more lines ...
    source_system = 'OLTP'
)

Then, simplify the main file repsales.prm:

REPLICAT repsales
...
MAP sales.orders, TARGET bi.f_orders,
OBEY map_orders_colmap.oby;

Benefit: Separation of primary and secondary logic. The main file clearly defines the “where from, where to” mapping relationship, while the specific transformation details are encapsulated in a separate module. Maintainers can quickly locate and modify the part they care about.

Scenario 4: Advanced Application - Nested OBEY and Environment Separation

When you have multiple environments like development, testing, and production, nested OBEY usage can achieve maximum configuration reuse.

Pain Point: How can you use the same parameter file template to manage environment-specific configurations (like database connection info)?
Solution:
1. Define Common Configuration: Create common_extract_config.oby containing parameters common to all environments (e.g., EXTTRAIL options).
2. Define Table List: Create app_tables.oby containing the tables to be synchronized.
3. Define Environment-Specific Configuration: Create a file for each environment.
  - prod.oby: USERIDALIAS ogg_prod_alias DOMAIN OracleGoldenGate
  - dev.oby: USERIDALIAS ogg_dev_alias DOMAIN OracleGoldenGate
4. Assemble the Main Parameter File: The main file extapp.prm is only responsible for “assembly.”
```
-- extapp.prm - Main parameter file for Application Extract
EXTRACT extapp
    
-- Include environment-specific settings (e.g., DB connection)
OBEY dev.oby  -- In DEV environment. Change to prod.oby for PROD.
    
-- Include common configurations
OBEY common_extract_config.oby
    
-- Include the list of tables to be extracted
OBEY app_tables.oby
```
Benefit: This achieves the ultimate form of “Configuration as Code.” 95% of the configuration (common settings and table lists) is identical across all environments, with only the environment-specific dev.oby or prod.oby being different. When deploying to a new environment, you just need to switch one OBEY call, greatly improving deployment efficiency and security.

Best Practices and Precautions

To maximize the power of OBEY files and avoid introducing new chaos, follow these best practices:

Establish a Clear Directory Structure: Don’t just throw all .oby files into the dirprm root directory. It’s recommended to create subdirectories, for example:
- dirprm/oby/tables/
- dirprm/oby/configs/
- dirprm/oby/maps/
Adopt a Naming Convention: Filenames should be self-explanatory. hr_tables.oby is far better than t1.oby.
Embrace Version Control: It is essential to put your entire dirprm directory under a version control system like Git. This gives you change tracking, code review, and quick rollback capabilities, which are the cornerstones of professional operations.
Add Thorough Comments: In the main file, add a comment next to each OBEY command explaining its purpose. The OBEY file itself should also have a header comment.
Beware of Two Pitfalls:
- Circular Dependencies: File A OBEYs File B, and File B OBEYs File A. OGG will detect this and throw an error, but it should be avoided in the design phase.
- Parameter Overriding: Parameters are parsed sequentially. If you define HANDLECOLLISIONS in an OBEY file and then define it again in the main file, the definition in the main file will override the one from the OBEY file.

Conclusion

The OBEY file is not an optional “advanced trick” but a core practice of professional OGG configuration management. By breaking down “monolithic” parameter files into modular OBEY files, we can achieve a qualitative improvement:

Core Value	Concrete Manifestation
Modularity	Logic is separated; configuration units are managed independently.
Reusability	“Write once, use everywhere,” avoiding repetitive work.
Maintainability	Change points are centralized, reducing complexity and risk.
Standardization	Enforce enterprise-wide standards via a configuration library.

The most important principle is: Treat your OGG configuration as code.

Starting today, review the largest and most complex .prm files in your environment. Try to extract the most obvious reusable part—like the table list—into an OBEY file. By taking this first step, you will embark on the path to more efficient and reliable Oracle GoldenGate management.

The Root of Chaos: The Drawbacks of Monolithic Parameter Files

Getting Started with OBEY Files: Basic Syntax and Principles

OBEY Files in Action: Four Classic Application Scenarios

Scenario 1: Centralized Table List Management (TABLE/MAP)

Scenario 2: Creating a Standardized Configuration Library

Scenario 3: Modularizing Complex Logic (e.g., COLMAP)

Scenario 4: Advanced Application - Nested OBEY and Environment Separation

Best Practices and Precautions

Conclusion

Scenario 1: Centralized Table List Management (`TABLE`/`MAP`)

Scenario 3: Modularizing Complex Logic (e.g., `COLMAP`)