[phisco]

Description of your changes

In this one-pager, I attempted to summarize the necessary context to understand the current implementation of the Composition Environment (also known as EnvironmentConfigs) as described by the original one-pager, and to discuss issues that we saw emerging with it.

Please, focus on the above first, so that we can have a shared understanding of what we are trying to solve.

In the end, I try to propose an approach to address the above issues, taking into account the overall direction Crossplane seems to be heading for, and to discuss the next steps we need before promoting this functionality to beta.

Please consider this section as a proposal, I decided to leave it there to have a starting point for discussion, but should be reviewed against any change to the context above and rewritten to be more "prescriptive" with whatever decision we take in the end.

I have:

• Read and followed Crossplane's contribution process.
• Run make reviewable to ensure this PR is ready for review.
• Added or updated unit tests.
• Added or updated e2e tests.
• Linked a PR or a docs tracking issue to document this change.
• Added backport release-x.y labels to auto-backport this PR.

Need help with this checklist? See the cheat sheet.

[jeanduplessis]

Suggested change

	- the concept of `in-memory environment` would just a convention between functions, relying on the [`apiextensions.crossplane.io/environment`](http://apiextensions.crossplane.io/environment) key in the context, as `function-patch-and-transform` already does.
	- the concept of `in-memory environment` becomes a convention between functions, relying on the [`apiextensions.crossplane.io/environment`](http://apiextensions.crossplane.io/environment) key in the context, as `function-patch-and-transform` already does.

[phisco]

everything else above is conditional, so I'd say:

Suggested change

	- the concept of `in-memory environment` would just a convention between functions, relying on the [`apiextensions.crossplane.io/environment`](http://apiextensions.crossplane.io/environment) key in the context, as `function-patch-and-transform` already does.
	- the concept of `in-memory environment` would become just a convention between functions, relying on the [`apiextensions.crossplane.io/environment`](http://apiextensions.crossplane.io/environment) key in the context, as `function-patch-and-transform` already does.

However I wouldn't put much focus on the form for this last part, as I want to rewrite it once the above context settles and we agree this is the way to go.

[lsviben]

Thanks @phisco for the great summary of the feature and future plans 🙌 . Added a few questions

[turkenh]

Great write up @phisco 🙌

I agree that it is very important that we are on the same page regarding what we are trying to solve before attempting to solve it 🙂 One thing that is not clear to me though, do you want to merge this without having the actual design or do you see this as a living document and plan to get it merged after adding details and once we reach to a consensus on the final design? Because I am not sure if it makes sense to aim merging it as a design doc without having the actual design 🤔

[turkenh]

Suggested change

	cross a single provider's boundaries.
	cross a single provider's boundaries and could only reference to a single, compiled type.

[phisco]

what do you mean by compiled?

[turkenh]

Just catching up with the latest state there. I need some time to process but my early thoughts are:

I am not yet convinced that the approach there should be the way we support composition environments while it might make sense for requesting extra resources. I feel like we need a more "native" support for environments rather than supporting them as a side effect (of capability of requesting extra resources) with a good amount of boilerplate code and 1-2 function calls each responsible with some part. This is mostly based on the fact that we know composition environments are crucial for any non-trivial composition and the desire to provide a smooth UX to enable this. So, my mind still goes to a solution like this instead of splitting this functionality to dedicated functions.

If I try to make an analogy:

We are designing UNIX and discussing how someone should pass some variables into commands and instead of defining/inventing the thing we know as Environment Variables in UNIX, we are talking about suggesting something like the following as a solution for those who need such a functionality:

env-requester | env-merger | cmd1 | cmd 2

instead of:

export VAR1=value1
source /path/to/somefile
cmd1 | cmd2

I know this is not the perfect example, so please don't list the differences between a UNIX environment and K8s/Crossplane etc 😅, the point I am trying to make is more about embracing a widely needed functionality and providing the best possible UX with native support vs "enabling" it somehow.

[phisco]

As per your example, the two things don't exclude each other, in shell scripts, piped commands have access to the environment.

To continue with the UNIX analogy, I see enabling functions to retrieve resources more like providing UNIX sockets and leaving to users to decide what to do with them.

What I'm suggesting is indeed, given that we have a finite amount of time, to put on hold the development of a native approach and instead focus on enabling functions. Nothing would block us from "bringing back in" as native the "best possible UX" once it emerges from functions and we agree on it.

[phisco]

@turkenh, I do want to rewrite the final part before merging, although it might be more a "plan" rather than a "design", as if we agree on the one proposed, the actual design will be needed for #4739, through its own one-pager, which I'm already working on; while here we'll only say we'll rely on that and take note of the needs we should take into consideration while designing that.

[phisco]

FYI, the proposal to add extra resource gathering capabilities to Functions: #5099

[phisco]

I think we reached consensus on the path forward here, see the last part of the proposal.

I understand that it's a weird Design Document. It's not actually proposing a design; it's more of an issue statement/documentation and a decision log. We clearly state that we won't be actively investing further into the native "Composition Environment" and won't set any timeline for its promotion to Beta. This might never happen if the function-based approach we think could replace the in-tree implementation works (NOTE: in such a case, we would offer as smooth a transition as possible from the native approach).

So, I still think it might be worth merging this just to have something to point people at when they ask when Composition Environment (a.k.a. EnvironmentConfigs) is going to be promoted to Beta or more trivial questions like "how to patch back to the EnvironmentConfig".

WDYT?
CC @MisterMX @negz

[negz]

@phisco Yeah we should definitely merge it.

It feels more like an ADR, which is something we wanted to adopt but never did #3602.

Maybe we could rename this to design/adr-00-???.md and merge it?

[phisco]

@negz, although I do like the idea in theory, in practice, I'm not a huge fan of ADRs. In my experience, they require a lot of discipline to get right, and it feels like there is a lot of overlap with the context already available in PRs and commits. But, maybe I just never saw them used as they should be; that's totally plausible.

I feel this could still be called a "one-pager" as I think it fits the definition provided by the design/README.md:

   A One-pager is a brief document pitching an idea for further
   experimentation. One-pagers, as the name suggests, are usually around one
   page long. They provide just enough context to achieve buy-in from Crossplane
   maintainers.

But the name of the file is definitely not a hill I would die on, so I could go either way.

[negz]

@phisco I don't feel super motivated to take on ADRs, so I'm happy to merge as is. I forget we'd defined what a one-pager was for, and I agree this does seem to fit within those bounds.