We often see people wanting to create technology that encodes certain values like fairness, justice, democracy, and so on. I want to reflect on how one might do this, especially for values that many people might understand at an intuitive level but have a hard time concretely defining. Ideally, we’d want to operationalize values like this in a way that’s tractable as an objective for a technological problem while also being faithful to the layman’s idea of what this means.
Importantly, when creating technology like this, formulating the problem is often more consequential than solving the problem itself. In particular, a formulation that operationalizes the value in question inaccurately risks creating a solution to a problem that may not exist — or worse still, a scenario that makes it more difficult for us to orient future solutions towards a more constructive operationalization.
Operationalizing privacy
My thoughts on this problem arose when I was doing my final year project (aka bachelor’s thesis) on how privacy might be preserved while training ML models. I chose this topic because I really wanted to work on a project that was at least somewhat related to considerations of technology’s wider societal impacts, and this provided an excuse to steer what was supposed to be a math project in this direction. Of course, any such work would require an operationalization of the notion of privacy — an idea that the vast majority of us have some conception of, and that most people probably also believe is an important consideration when building technology, but is also not the easiest thing to pin down precisely in a setting like this.
One of the most common ways of operationalizing privacy in a mathematical setting is differential privacy (DP). Informally, this provides the following guarantee when applied to training an ML model:
Suppose I train an ML model on two datasets that have exactly the same set of data except for one entry: one has your data and one doesn’t. The model is said to be trained in a differentially private manner if after doing so, you cannot tell if your data has been included in the training set or not by just analyzing the model’s output.
Traditionally, differentially private training is done using an algorithm called DP-SGD, and membership inference attacks can be used to test if the privacy guarantee from the training does indeed hold.
Is differential privacy real privacy?
However, as my project wore on, I began to feel that DP is a poor operationalization of privacy when applied to ML in the present-day (particularly when applied to LLMs), and this will increasingly be the case as the field moves forward. This is for three reasons:
Neural network architectures and training methodologies have gotten much more complex since DP-SGD (and its variants) were first proposed. For example, when DP is applied to LLMs, privacy guarantees should be maintained through all of pre-training and post-training. This post highlights how DP fine-tuning’s guarantees are complicated by how similar the fine-tuning and pre-training datasets are, as well as how capable the LLM in question is.
We know that the LLMs of today are trained on what is essentially the entire internet. Thus, DP’s threat model becomes rather questionable, because we know that the answer to “has my data been included in ChatGPT’s training set?” is most likely (or will be in its next iteration) a resounding yes.
LLMs provide new threat vectors that can be more potent than simply including sensitive information in their training data. For example, Lermen et al. (2026) find that LLM agents can be used to identify users from anonymous online posts by matching profiles across different platforms like Reddit and LinkedIn. It’s worth noting that a model trained perfectly well using DP would still be able to use its capabilities to carry out such identification. This suggests that a new operationalization of privacy that encompasses more than just an ML model’s training data is in order.
In addition, Brown et al. (2022) and Mireshghallah & Li (2025) have both written extensively about privacy issues and LLMs in much more detail. Beyond LLMs, it’s also likely the case that any other ML models (especially general-purpose ones) that use a sufficiently large amount of data in their training will surely suffer from similar privacy issues.
What might it mean to operationalize privacy differently in the context of training ML models? One example would be to have hardware-based privacy guarantees by doing training inside trusted execution environments (TEEs). Workshop Labs (now part of Thinking Machines) previously created a stack that allows you to do this for post-training, which makes it possible to fine-tune a customized model based on your own data without there ever being a need for it to pass through a third party channel that might introduce the risk of a leak.
Constant vigilance
One might be tempted to think that to prevent us from solving problems that operationalize values inaccurately, we should spend as much time as possible thinking through all possible definitions, implications, and other considerations before formulating and solving the problem. However, in many cases it’s not possible to pre-emptively enumerate every single thing that could affect the validity of the operationalization, as some of these factors might only arise once build your solution and see how it interacts with everything else (after all, the real world is complicated and messy).
Instead, it’s more important to always be on the lookout for potential changes that might make your operationalization more accurate. For example, DP was originally created as a privacy-preserving tool for obtaining statistical queries from databases. For this, it remains (to me at least) a perfectly reasonable operationalization of privacy. But we have to be careful when applying it in other contexts, where there might be aspects of privacy that aren’t captured by DP itself, especially when they come from sociotechnical considerations that are sometimes easy for technologists to miss. We shouldn’t have our imaginations of what our values entail constrained by a convenient mathematical formulation just because it already exists.
Thus, I think this problem of operationalizing values is of utmost importance for anyone who wants to build technology “for good”. When you have such lofty ambitions, the devil is often in the details in ensuring that these values are actually embedded in our technological solutions. And as we create increasingly powerful and general-purpose technology, I expect that we’ll need to be even more careful when treating this issue of operationalization.