The Mechanics of Crossplane: Providers, Compositions, and Pipelines
Following my Introduction to Crossplane, it’s time to look under the hood. To build a production-grade internal developer platform (IDP), you need more than just definitions; you need a way to manage cloud resources at scale without drowning in boilerplate. This is where Providers, Compositions, and Pipelines come in.
1. Providers: The Resource Bridge
If you think of Crossplane as a Kubernetes operator, Providers are the specific workloads that handle the "talking" to external systems. They provision Managed Resources (MRs)—high-fidelity, declarative versions of cloud resources like AWS S3 buckets, Github repositories, or GCP databases.
Installation is standard Kubernetes YAML:
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
name: provider-aws-rds
spec:
package: xpkg.upbound.io/upbound/provider-aws-rds:v1.0.0
Applying this creates a new set of CRDs and a controller pod. But how do you handle credentials? You use a ProviderConfig. You can have multiple configs in one cluster (e.g., one for dev and one for prod), allowing your resources to target different environments just by referencing a different config name.
2. Compositions and Pipelines: The Logic Layer
In the past, Crossplane logic was limited to "Patch-and-Transform"—simple mapping from a Claim to an MR. While functional, it lacked the procedural power needed for complex tagging, conditional logic, or external lookups.
The modern answer is the Composition Pipeline. It treats resource generation like a sequence of steps, where each step is a Function. The output of one function becomes the input for the next, similar to a Unix pipe (|).
The Pipeline Context
This is the "shared memory" of your pipeline. A function early in the pipeline can fetch data (like your company’s standard cost-center tags from a ConfigMap) and store it in the context. Later functions can read that data and inject it into every resource being created.
pipeline:
- step: fetch-tags
functionRef: { name: function-extra-resources }
- step: render-resources
functionRef: { name: function-cue }
3. Why This Wins
The pipeline architecture solves the biggest headache in IaC: standardization. By enforcing logic in the control plane rather than in templates, you ensure:
- Standardized Tagging: Cost tracking becomes invisible to the developer and 100% accurate for management.
- Compliance by Default: Security groups, IAM roles, and encryption settings are injected automatically by the platform team’s functions.
- Velocity: Developers request a "Database," and the platform handles the complexity of where it goes and how it's secured.
In the next post, I’ll document my experience writing a custom Go-based function to handle dynamic IP allocation—a real-world use case that would be a nightmare in traditional templates.