On conviendra aisément qu’il importe au plus haut point de savoir si l’on n’est pas dupe de la morale. [Everyone will readily agree that it is of the highest importance to know whether we are duped by morality.]

Totality and Infinity, E. Levinas

You and I imagine different things when we imagine a neural network, a multilayer perceptron like those that grace the newest wave of machine learning systems. Like nearly all neural network people (you don’t have to be one of them to read this essay, but it will help), I do not imagine neurons at all. When I imagine a neural network I see a mathematical machine, an optimization machine. But unlike nearly all neural network people, I do also see something different when I imagine a neural network. In addition to matrix multiplications, nonlinearities and layers of transduction of gradients, I see a society. I see a society of agents interconnected to each other by relations between them, each node an agent, each edge a relation.

This is not the society that earlier connectionists saw, the society of mind. I see a brute, hard-headed, money-grubbing society. I see a society where coin is traded, where prices are set, where credit is assigned in a very literal way and the credits and weights are measured in hard currency.

But I was surprised when I looked at the problem of alienation in such a society. I saw that the problem of alienation had a formal solution in that algorithmic society. The solution was not presented simply in clear language. It is not original to me. But the problem, if you have my point of view, was clearly posed. It was so clearly posed that it was solved. That solution is the subject of this essay. I do not know whether that solution will translate to real groups of people. I do not know whether unforeseen circumstances will destroy the practicability of this solution. I suspect you will be interested in reading that solution anyway.

The Problem

Doctors without borders said that people needed 7 days practical training on PPE before sending them into an ebola treatment unit. We would give illiterate west africans 200 page powerpoint presentations and then let them try on the equipment once before sending them in. Most of them died. But it was ok, because look at how much work we (WHO) are doing here! We are training and sending in HUNDREDS of workers! And they are all dying!!! This means the outbreak is unstoppable! Even if we had done a good job at the start (which we didn’t) it wouldn’t have made any difference. So don’t blame us now.

If you have ever seen “generation kill” that is exactly how it went down. Every day we would be all about “impressing geneva”.

Boss: are you doing anything important?

Me: Yes, i’m figuring out the ebola village hotspots for the last 3 days to prioritize the teams

Boss: No, (district) just came out with a powerpoint that is really impressing Geneva. You need to recreate it for our district.

Me: And the hotspots/team directions?

Boss: Leave it.

Anonymous [verbatim]

But actually, he thought as he re-adjusted the Ministry of Plenty’s figures, it was not even forgery. It was merely the substitution of one piece of nonsense for another. Most of the material that you were dealing with had no connexion with anything in the real world, not even the kind of connexion that is contained in a direct lie. Statistics were just as much a fantasy in their original version as in their rectified version. A great deal of the time you were expected to make them up out of your head. For example, the Ministry of Plenty’s forecast had estimated the output of boots for the quarter at one-hundred-and-forty-five million pairs. The actual output was given as sixty-two millions. Winston, however, in rewriting the forecast, marked the figure down to fifty-seven millions, so as to allow for the usual claim that the quota had been overfulfilled. In any case, sixty-two millions was no nearer the truth than fifty-seven millions, or than one-hundred-and-forty-five millions. Very likely no boots had been produced at all. Likelier still, nobody knew how many had been produced, much less cared. All one knew was that every quarter astronomical numbers of boots were produced on paper, while perhaps half the population of Oceania went barefoot. And so it was with every class of recorded fact, great or small. Everything faded away into a shadow-world in which, finally, even the date of the year had become uncertain.

Nineteen Eighty-Four, G Orwell (EA Blair)

… As the end of the 1968 fiscal year approached, Trans World Airlines (TWA) was headed for a major financial loss after recording sizable profits in 1967. In response to the crisis, TWA’s managers took action: They “extended the depreciable life of most of [the airline] fleet by several years and took down more of [the] available investment tax credit in computing deferred income taxes” (Mason and Swanson, 1981, p.137). When the ink was dry, TWA had gone from a near certain substantial loss to a profit equal to more than half of that of the previous year. Managers whose pay was based on the company’s annual profit appreciated the change more than financial analysts trying to evaluate the health of the firm.

Measuring and Managing Performance in Organizations, RD Austin

No aspect of the problem of alienation is unknown or unstudied. However, I do not believe that the fact that there is an algorithmic approach is widely known or construed to be known. But it is still worth making a recapitulation of the problem in light of the purported solution, even if that recapitulation is unoriginal.

Alienation is a phenomenon where there is a distance between entities which should be together. In that definition, it is “should” which shoulders the burden. That is because without a system of credit assignment, there cannot be alienation. By a system of credit assignment I mean a system by which people make judgments, whether it be by money, by acumen, by political savvy or by bureaucratic might. These judgments involve the weighing of choices by credit or debit, by good or ill. When we talk of alienation, we must always talk of systems of credit assignment, both in the abstract and in the concrete. The distance of alienation is a distance measured in credit assigned.

It is particularly easy to see and study systems of credit assignment in money. This is because there, credit is denominated simply and in currency. Looking at the money or pointedly not looking at the money has been the approach taken by the great theorists of alienation. But it is easier yet to see such systems of credit assignment in the neural network. There every aspect of the system can be brought to light. Such and such an agent in the network contributes such and such a weight to another node, which in turn contributes this weight.

The great impact that alienation has upon real systems of credit assignment is because, when it occurs, it always co-occurs with a phenomenon which I call credit eutrophication, in analogy to ecological eutrophication. Ecological eutrophication goes thus. If one dumps too many phosphates or nitrates in freshwater lakes or rivers, great blooms of algae take up these nutrients. This is because the algae grow fastest and have the largest capacity to take up the nutrients. These blooms of algae proceed to die quickly and the bacteria that make them rot take up the rest of the resources in the river, including oxygen. The surface of the water is then covered with choking and dead fish.

The analogy to credit eutrophication is straightforward. The credit assignment in systems still goes on even when the credit is being withheld from the alienated. That credit just goes to those agents in the system which have a larger capacity to take up credit. We cannot do anything about the Ebola, because we must write the reports. The shoe production figures are great, who cares about the actual number of shoes? This is the source of the great importance of alienation, because of the great mis-allocation of resources that it creates.

Modernity is the age of alienation. It is the age of alienation because there is so much of it. There is so much of it because there is a proliferation of a difference in the structure of credit assignment. That difference is uniquely conducive to alienation. That difference is merely that there are many layers of credit assignment.

By a layer I mean something analogous to a trophic level in a food web. I also mean something analogous to a layer in a neural net. The insect eats the algae, the fish eats the insect, but the hawk does not eat algae, except by way of the fish. The subsistence farmer eats the produce of their own making, but the commodity farmer eats from the supermarket. The supermarket takes the food from the factory, the factory takes the food from the grain mill, the grain mill takes it from some other commodity farmers. Modernity is the age of such supply chains, the age of multiple steps of production.

The key to the problem is that the chain of credit assignment is not equal in its ability to soak up credit, just like the trophic levels are not equal in their ability to soak up nutrients. And in soaking up credit and therefore resources, they deprive the other layers of necessary resources. The same inequality occurs for agents within layers in a system.

The inequality comes from the fact that in order for the credit to be transduced through the system, each set of relations between layers must be in concordance, like a series of soft AND gates, or like a matrix exponentiation as in the vanishing and exploding gradient phenomenon. This is the source of the dual presented previously between the eutrophied and alienated state, because a very few parts of the system have credit transduced towards them, and nearly every other part of the system does not, including very often the overall purpose of the system.

To use some of the jargon, these are instances of the generalized assignment problem, but instances of the generalized assignment problem closer to the critical phase.

Often the credit assignment goes to something very replicable and very easy to pour credit towards. Documents, ideologies and other invisible tools have this property. Chief among invisible tools is money. You may know the name some certain philosophers have given to this phenomenon.

This is the problem with coloring certain portions, certain layers or certain agents of the system and declaring that they are the key to some cure for alienation. They then become ideal targets for eutrophication. This is the key irony of systems like the Chinese social credit system, which does not have a systematic answer to that eutrophication and therefore becomes subject to it immediately. That is, the Party leadership itself is not subject to the social credit system. Therefore, by adding that layer of abstraction it will only serve as another layer of credit assignment, an inducement towards alienation, towards credit eutrophication, and towards the misallocation of resources.

An aside. Note that one cannot have technological matters be moral matters but moral matters never be technological matters. Unless you are Levinas there must come logical and ontological preconditions to ethics. That is, you may have certain moral positions, but the definitional structure of the world is a precondition for having moral positions at all. You must know that spam is possible in the world before you know that spam is bad. You must have a definition for phishing before you can claim that phishing is unethical. Not only artifices of high technology: that money can be measured in numerical form, that credit can be written down and the memory of it preserved, and that judgment can be made from writing, these are technological contingencies, but they affect our ethics.

System

A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system.

General Systemantics, J. Gall

The problem of alienation is thus posed. If you believe me, then we can go ransacking the tools of neural networks to attack the problem of alienation. The AI folks have these tools, and the economists and politicians do not, because you can set up an AI experiment with a few computers at 2 AM in your pajamas without permission from anybody else and mostly without ethical concerns. The experiment finishes in less than a few days or weeks, and replication is, comparatively, a cinch. Validation is less good.

Many people inveigh against the dire state of understanding of neural network systems. This is because they expect an understanding comparable to the dynamicist or kinematist. To the level available to the statistical mechanic or chemist, the understanding is often suitable. To the level available to the economist or bureaucrat, the understanding is remarkable.

The specific implementation will come in a model inside of a software apparatus. The model is to be synchronized with the actual credit flows of the model in a way analogous to a distributed version control system. This will entail having a credit commit log, in analogy with the ordinary version control commit log, with atomic instances of credit assignment recorded in a ledger. The salesman makes the sale, the buyer makes a purchase, enter it into the ledger as is currently done.

The difference from an ordinary ledger comes in the treatment of the agent who is not directly entering into agreements. It is often held to be impossible to talk of judgment of agents who are not directly related to direct credit assignments in an organization, but the judgment is actually simple in this formalism. Just backpropagate. Every so often, the relations of production will be adjusted (“learned”) via second order backpropagation, the second order used for arcane technical reasons.

By second order methods I mean the usage of ordinary Newton’s method as the credit assignment algorithm, instead of the more usual gradient descent. Usually the reason given for not doing this is the computational cost. However, I found a way to make it practicable for reasonably small networks a few months ago.

The extremely technical hypothesis about second order methods is that they renormalize the credit assignment flow. At each layer, the flow would be prone to matrix exponentiation in gradient descent, but at each layer in the second order method, there is also a computed matrix inversion which very literally renormalizes the flow. Think of a scalar exponential series versus a scalar harmonic series. The harmonic series will decay but it still diverges. The length of the correlational structure and the extension of the credit, therefore, would diverge to arbitrary numbers of layers of credit assignment. This is why I believe that this will be the most fruitful approach.

More important and more pervasive will be the attempts, systematic and unsystematic, to lie to the system. The perceptive will note that in talking about the misallocation of credit to documents and ideologies, I have created a document and perhaps an attendant ideology.

The picture of the firm or bureaucracy as a system wherein the agents are trusted is a picture only. It becomes quite the fiction as the firm gets bigger. The answer is not to create some monstrosity of high modernism. It is to start with a small system dealing with a small, low-impact part of a small domain and deal with the problems as they come by when they are small, as they will. Principled and systematic rules are mostly results, not causes.

The specific countermeasure I will start with will consist of another model which will be trained on examples of agents, inputs, outputs and sections of the system with fraud. But first, these examples must exist, so it will be impossible to immediately deploy a solution to large organizations. The phenomenon of renormalization group flow is also closely related to Benford’s law, which has also previously been used to attack fraud.

If one is inclined to think about algorithmic mechanism design, I believe that the model corresponds to the hypothesis that the firm or organization is a Walrasian auction with recursive substructure.

I must now confess that I hid the actual purpose of the essay from you. This essay is prolegomenon to the actual implementation of a system which includes these ideas. I have several prototypes of the system which I will not release because I know what happens to prototypes. I have begun work on a release implementation. I said the first version could perhaps be done by the end of this year. This deadline will not be met. But I will continue.

Thanks to JB and others for reading.