Applications Overview¶

Applications in OpenMetadata represent software systems and tools that interact with the metadata platform - consuming metadata, automating governance workflows, extending OpenMetadata capabilities, or integrating metadata into other systems. These are operational applications that leverage OpenMetadata's APIs and events to build metadata-driven solutions.

What are Applications?¶

Applications in the OpenMetadata operations context are systems that:

• Consume Metadata: Read entity metadata, lineage, quality metrics via APIs
• Automate Governance: Apply tags, assign owners, enforce policies automatically
• Extend Capabilities: Build custom features on top of OpenMetadata
• Integrate Systems: Sync metadata with catalogs, CMDB, ITSM tools
• Enable Workflows: Trigger actions based on metadata events and changes
• Build Custom UIs: Create specialized interfaces for metadata consumption

Application Categories¶

Metadata Consumers¶

Applications that read and use OpenMetadata: - Custom data catalogs and portals - Metadata-driven dashboards - Data discovery applications - Compliance reporting tools - Cost optimization tools

Governance Automation¶

Applications that automate metadata governance: - Auto-tagging and classification engines - Policy enforcement applications - Data quality orchestrators - Ownership assignment tools - Compliance validators

Integration Applications¶

Applications that sync metadata across systems: - ServiceNow CMDB integrations - Jira ticket automation - Slack/Teams bots - Corporate catalog sync - Cloud platform integrations

Application Types¶

OpenMetadata tracks various application types:

Web Applications¶

• Customer-facing websites
• Internal portals and dashboards
• Admin interfaces
• Single-page applications (SPAs)

Mobile Applications¶

• iOS and Android apps
• Progressive web apps (PWAs)
• Hybrid mobile applications

Microservices¶

• REST APIs
• GraphQL services
• gRPC services
• Event-driven services

Backend Services¶

• Authentication services
• Payment processing
• Email and notification services
• Search and recommendation engines

Data Applications¶

• ETL/ELT tools
• Data ingestion services
• Stream processing applications
• Batch processing jobs

Analytics Applications¶

• Business intelligence tools
• Reporting applications
• Data visualization platforms
• Analytics engines

Application Entities¶

graph TB
    A[Application] --> B1[Components]
    A --> B2[Data Dependencies]
    A --> B3[API Endpoints]
    A --> B4[Configurations]

    B1 --> C1[Services]
    B1 --> C2[Modules]
    B1 --> C3[Libraries]

    B2 --> D1[Databases]
    B2 --> D2[APIs]
    B2 --> D3[Files]
    B2 --> D4[Streams]

    style A fill:#4facfe,color:#fff,stroke:#4c51bf,stroke-width:3px
    style B1 fill:#00f2fe,color:#333
    style B2 fill:#00f2fe,color:#333
    style B3 fill:#00f2fe,color:#333
    style B4 fill:#00f2fe,color:#333
    style C1 fill:#ffd700,color:#333
    style C2 fill:#ffd700,color:#333
    style C3 fill:#ffd700,color:#333
    style D1 fill:#764ba2,color:#fff
    style D2 fill:#764ba2,color:#fff
    style D3 fill:#764ba2,color:#fff
    style D4 fill:#764ba2,color:#fff

Application Metadata¶

OpenMetadata captures:

• Application Properties: Name, version, repository, deployment
• Technology Stack: Programming languages, frameworks, libraries
• Data Dependencies: Databases, tables, APIs, files, topics consumed
• Data Production: Data created or published by the application
• API Contracts: Endpoints exposed and consumed
• Infrastructure: Hosting, compute resources, scaling configuration
• Team Ownership: Development team, product owner, SRE team
• Documentation: Architecture, runbooks, API documentation

Use Cases¶

Data Governance¶

Understanding which applications access sensitive data: - Track PII and regulated data usage - Enforce access policies - Audit data access patterns - Manage compliance requirements

Impact Analysis¶

Assessing the impact of data changes: - Identify affected applications when schemas change - Understand downstream dependencies - Plan migration strategies - Prevent breaking changes

Data Lineage¶

Mapping data flow through applications: - Track data transformations - Understand end-to-end data pipelines - Identify data sources and destinations - Validate data accuracy

Cost Management¶

Optimizing data costs: - Identify high-volume data consumers - Optimize query patterns - Right-size resources - Reduce redundant data access

Security & Compliance¶

Managing application security: - Track credential usage - Monitor data access patterns - Enforce encryption requirements - Audit application permissions

Application Architecture¶

graph LR
    A[Client Apps] --> B[API Gateway]
    B --> C[Microservices]
    C --> D1[Database]
    C --> D2[Cache]
    C --> D3[Message Queue]
    C --> D4[Object Storage]

    E[Background Jobs] --> D1
    E --> D3
    E --> D4

    F[Analytics Platform] -.->|reads| D1
    F -.->|reads| D4

    style A fill:#43e97b,color:#333
    style B fill:#667eea,color:#fff
    style C fill:#4facfe,color:#fff,stroke:#4c51bf,stroke-width:3px
    style D1 fill:#764ba2,color:#fff
    style D2 fill:#764ba2,color:#fff
    style D3 fill:#764ba2,color:#fff
    style D4 fill:#764ba2,color:#fff
    style E fill:#f093fb,color:#333
    style F fill:#ffd700,color:#333

Data Contracts for Applications¶

Applications can define and enforce data contracts:

Consumer Contracts¶

What data the application expects: - Schema requirements - Data quality expectations - Freshness requirements - SLA agreements

Producer Contracts¶

What data the application provides: - Output schema guarantees - Quality commitments - Update frequency - Deprecation policies

Integration Points¶

Applications integrate with various data assets:

graph TD
    A[Application]

    A -->|reads from| B1[Tables]
    A -->|consumes| B2[Topics]
    A -->|calls| B3[APIs]
    A -->|reads| B4[Files]

    A -.->|writes to| C1[Tables]
    A -.->|publishes to| C2[Topics]
    A -.->|exposes| C3[APIs]
    A -.->|generates| C4[Files]

    A -.->|uses| D1[ML Models]
    A -.->|queries| D2[Dashboards]
    A -.->|executes| D3[Stored Procedures]

    style A fill:#4facfe,color:#fff,stroke:#4c51bf,stroke-width:3px
    style B1 fill:#764ba2,color:#fff
    style B2 fill:#764ba2,color:#fff
    style B3 fill:#764ba2,color:#fff
    style B4 fill:#764ba2,color:#fff
    style C1 fill:#00f2fe,color:#333
    style C2 fill:#00f2fe,color:#333
    style C3 fill:#00f2fe,color:#333
    style C4 fill:#00f2fe,color:#333
    style D1 fill:#f093fb,color:#333
    style D2 fill:#f093fb,color:#333
    style D3 fill:#f093fb,color:#333

Governance Practices¶

Application Registration¶

• Centralized application catalog
• Ownership assignment
• Technology stack documentation
• Dependency mapping

Access Management¶

• Service account tracking
• Credential rotation policies
• Least privilege access
• Regular access reviews

Change Management¶

• Impact assessment for changes
• Breaking change notifications
• Deprecation policies
• Version management

Monitoring & Observability¶

• Data access monitoring
• Performance metrics
• Error tracking
• Usage analytics

Best Practices¶

1. Document Dependencies¶

Maintain a clear catalog of all data dependencies.

2. Use Data Contracts¶

Define explicit contracts for data consumption and production.

3. Version APIs¶

Use semantic versioning for application APIs.

4. Implement Graceful Degradation¶

Handle data source failures gracefully.

5. Monitor Data Quality¶

Track quality of consumed and produced data.

6. Log Data Access¶

Maintain audit logs of data access patterns.

7. Automate Discovery¶

Use automatic dependency discovery tools.

8. Regular Reviews¶

Periodically review and update application metadata.

Application Lifecycle¶

graph LR
    A[Development] --> B[Testing]
    B --> C[Staging]
    C --> D[Production]
    D --> E[Deprecation]
    E --> F[Retirement]

    style A fill:#ffd700,color:#333
    style B fill:#00f2fe,color:#333
    style C fill:#f093fb,color:#333
    style D fill:#43e97b,color:#333
    style E fill:#fa709a,color:#fff
    style F fill:#999,color:#fff

Related Entities¶

• Application: Detailed application entity specification
• Table: Database tables accessed by applications
• API Endpoint: APIs called by applications
• Topic: Message streams consumed/produced
• ML Model: Models used by applications
• Pipeline: Data pipelines for application data
• Data Contract: Contracts between applications and data
• Team: Teams owning and operating applications

Next Steps¶

• Application Entity: Detailed specification for application entities
• Data Contracts: Learn about data contracts for applications