Why I don't mind training "AI"

This reminds me of an event from 2010: In the 2000s I collaborated with Jonathan Duddington on eSpeak before he passed away and they forked it into eSpeak-NG, and in 2010 a Google engineer emailed us to say his team had used eSpeak as a voice for Google Translate for some languages---it was live for most of that year until they found a commercial replacement. At about that time someone in the Engineering department asked me if I could demonstrate any of the software I had worked on, so I opened Google Translate and played the voice, whereupon he exclaimed "that's not you---you got it from Google" and I had to say "no, Google got it from me" (well mostly Jonathan but I helped). Similarly in the mid-2020s I have several times been falsely accused of LLM plagiarisation by people who observed only the style and I had to say "people like me basically are the language models" because we did it first.

You won't get far asking the LLMs themselves about their training data because that's like asking a human for exact details of events that helped their brain learn to walk: they probably won't have a clear episodic memory of the whole thing, just a set of weights that represents an "average" of all the input they got, tuned by people working for the company to bias it in favour of the eventual behaviour they want, or at least behaviour that tends to make the human trainers tick the "good" box, which might in some cases not actually be what we need, because training workers are flawed and that's how you get problems like sycophancy. Against a background like this, a random article I wrote that was processed during pre-training would probably be about as "memorable" as a toddler's seeing me across the street while in the charge of adults who are letting them see the world but directing them somewhere specific. While it does have non-zero effect on the developing brain, and in some rare cases might even be retained enough for specific details to be recalled much later, it's far more likely mostly to be reinforcing a learned intuition of how people on the street tend to behave.

Some creators become upset that their output is used to train models, and I believe they have every right to be in control of how their output is used, but I don't personally understand why they're as anxious as they are. Yes, I would definitely be concerned if an LLM were to train on my private data like my inbox: this may contain details which other people have entrusted to me, and I would be betraying such trust to leak the data anywhere they weren't expecting: even though the probability that a model will overfit its training data badly enough to recall a specific detail from my email is very small, it is non-zero, and it would be reckless of me to say "oh let them train on my private data because the LLM probably won't remember the details" (for the same reason I avoid extreme sports---the survival rate may be high but the risk is still unnecessary). But public data is a different thing entirely.

Public data---web essays, free/libre and open source code, that sort of thing---is data I want anybody to be able to use if it's useful, and it doesn't particularly bother me if some of the links between myself and the end-user involve large for-profit companies. For example, I have written accessibility tools that can be used by people with visual problems like myself, and I want the other people with visual problems to benefit from these. If that's the goal I'm focused on, it doesn't really bother me that:

• there are multiple commercial for-profit Internet service providers involved in copying the bytes from me to the end-user, and by putting my bytes on the Internet I am contributing to the commercial value of those providers without any direct returns;
• the end-user is likely to locate my collection of bytes by means of a large Internet search engine which is probably being run commercially, and allowing my data to be indexed by that search engine is in effect a small extra contribution to its huge commercial value without any direct returns;
• the end user themselves might be working as a consultant or earning a high salary somewhere, some of which they need to do for a living but I'm not policing whether they're earning more than they need or what they do with the rest of it, and by making my bytes available for their general use I have also probably made a contribution to the value of whoever employs them, without any direct returns (in fact I specifically migrated my code from the GPL to the Apache license whenever possible, mostly because some misguided workplaces were refusing to allow even internal use of GPL and I felt sorry for the struggling employees there);
• and now that LLMs exist, I'm not sure why it should bother me if the mechanism by which my bytes benefit the end user may now involve an LLM intermediary, even one developed and/or run by a large for-profit company, to which I have therefore contributed without any direct returns.

There's an argument that I probably should try to make as much of my public data as possible available to LLMs for the purpose of at least trying to balance out the biases such systems can pick up if they are not trained on input from a diverse enough selection of people, especially if such biases are likely to pass unnoticed, precluding any "just let it be wrong so the problem is more obvious" argument---previous work trying to encourage manufacturers and big software houses to apply universal design showed me that trying to let something big be "obviously wrong" doesn't usually work as well as contributing an improvement for 'minority' users. (Yes I was one seat away from Rana el-Kaliouby when we were PhD students in the computer lab and yes this became her AI-training philosophy, go Rana!)

I do sometimes get a little concerned with what happens to my public data: I don't think it would benefit the end-user so much if, for example, they are given a really outdated old version of my code containing bugs I've long since fixed, so if I notice old copies of my work floating around I do tend to push that these be brought up-to-date (keeping a history is OK but I still prefer there to be at least a clear link to the current version), or else taken down so they don't clutter search results for the up-to-date version, but I make this a 'soft' request as it's not a big enough concern to warrant adopting a reputation as a draconian enforcer---the search engines usually do manage to list the current version first. Similarly I'd be a little concerned about LLMs learning something "wrong" from my work---and it wouldn't be very nice to the planet if I could say "hey Moonshot, Anthropic, OpenAI et al you're going to have to delete your entire training run and start it again because I've just fixed a typo in my web essay you crawled last week" (well there is some research into resource-saving incremental updates, but they wouldn't be able to do even that just whenever you email them), but this concern is tiny because as I said the influence of each part of the training input is not all that big. Modern LLMs are veering towards grounding themselves with actual search results anyway.

Also, sad to say there exist far worse things on the Internet than any mistake I'll ever write. Any "imposter syndrome" I experience from thinking my output is somehow "not good enough" to be used can usually be addressed simply by reminding myself that the user's other alternatives may be even worse, so I start thinking "even I can do better than that" instead.

Yes, "overfitting" is a concern: I wouldn't want a model to go wrong by 'regurgitating' something I'd written exactly, in a context where it's not helpful to do so. That's why I didn't like Getty Images copy-pasting my description of the Xu Zhimo stone without attribution in 2018 (removed in 2025): if my words had been generic enough to actually fit in their catalogue I wouldn't have minded, but in writing them for my home page I had subtly overemphasised a personal connection with the subject, which I thought was appropriate on my personal home page but became a potentially misleading implication when taken out of that context and put into a third-party catalogue---I didn't want the whole world to think Xu Zhimo liked Thomas Hardy more than any other poet if I had not established this by research---and the feeling of "hey, if I'd been writing for your site I wouldn't have worded it like that" was my main motivator for wishing they'd respected my copyright (even though that might not have been the original purpose of copyright) or at least given me an attribution so the reader might be prompted to check the original context if the exact words matter to them. But if an LLM tends to overfit then that LLM has bigger problems than the use of my particular output.

Yes, "fair" monetary compensation does seem like an unsolved problem, but this is nothing new to LLMs---it has always been the case that, as ex-NASA engineer and cartoonist Randall Munroe put it, "a project some random person in Nebraska has been thanklessly maintaining since 2003" has propped up infrastructure, including commercial infrastructure, and whatever the solution is, I'm not sure that just telling random entities they "can't use it" helps---do that and you still won't get paid, as they'll simply find other things to use, and you missed out on being able to have some small influence on making the system less bad for people who might need it.

The problem with making a stand against "AI training" on our data is, as I understand, the negatives of such a stand outweigh the positives. If we believe the meat industry is harming the planet and/or animal welfare, then we can stop eating meat and encourage others to stop, and however small a difference we make will, assuming the meat industry's stock-control systems are actually working, eventually translate back up the supply chain to fewer animals being bred for slaughter and less associated resource consumption. Similarly we can take reasonable measures to use less energy and expect this eventually to propagate back up the supply chain to less pollution etc. But trying to take steps to avoid our data being used for training LLMs doesn't really work like that, because the effect on the LLM's final quality is not readily measurable: yes if we could somehow coordinate everyone to refuse access to their content simultaneously then we might be able to get the big companies to sit down at a negotiation table, but blocking out a percentage of it one little bit at a time, even if we managed to persuade enough people to block out a substantial percentage of it, would not realistically achieve much, and meanwhile it may erode diversity of input in a way that could hurt minority users without being noticed by the executives we were trying to influence.

Traceability would be nice too: many people publish free software thinking "if I can't be paid for this work, at least it makes a good advertisement for me as a worker when I'm applying for jobs or whatever" and removing their credit can undermine this, which is why I wouldn't want to republish anybody else's work without at least trying my best to credit them, and free licenses usually do require this. Even if the credit is not at all obvious, it should at least be possible for the original author to be able to prove how they contributed, if they ever needed to make such a proof in front of a potential employer or something. Recall the "did I get it off Google or did Google get it off me" incident---in that case I had a private email and public blog post from a Google engineer I could have pointed to if he'd persisted, but it can be less easy to show if your work has helped some large corporation indirectly via an LLM.

At least search-grounded responses might make it easier for LLM users to check credit (if they will) for anything that contributed especially significantly, and I'm not particularly worried about uncredited trivial contributions---what would be the point in instituting a system more finicky than the pennies paid to obscure music publishers for occasional use of their works in small settings? Some few might benefit more, but mostly it would slow everybody down without helping, unless perhaps somebody holds the rights to a "long tail" of data. Copyright law already allows activities like indexing and lexicography---you don't have to chase down your contribution to helping someone build a dictionary of how words are used---and LLM training strikes me as just a fancier version of dictionary compilation.

Similarly, if I taught a child something which they then grew up and used in their work as a consultant, I'd be happy I'd helped their career; I wouldn't expect anything back just because the parents of that child were spending a lot of money on their upbringing while I had provided my small input without charge, or because the adult they became managed to earn more money than I ever did with my ideas. This isn't a direct analogy because LLMs are not people, but in both cases there is a limit to how much it makes sense to trace down every last little use of my every thought: blatant copying of large chunks of it that I hadn't licensed could be a problem, but being a tiny contributor to what is essentially like a lexicography project in my mind would not. (Disclosure I do have my own small lexicography project.)

Yes I have experienced periods of unemployment longer than I'd like, but "AI" is just one more excuse in a long line of them, and I don't think refusing to allow my public work to be used for training can possibly improve that situation for anybody. I do understand why people may want to exert more control over works they produce for commercial sale rather than gratis distribution, especially if they have a unique style which they'd rather avoid models being able to imitate, but even then I'm not entirely convinced censoring their works from the training data has as much effect as they think (but I don't want to get in the way if that's their decision; I'm just explaining why it's not mine).

Also, I certainly do understand the annoyance of small server operators if "AI" crawlers increase their traffic too much: that's a problem that needs to be solved, but "please don't hammer this small server" is different from "please don't use my data for training"---there are often ways to copy data to large repositories (usually Git-driven) where it can be used for training without your having to pay their traffic bills; if you wouldn't mind that then your server traffic issue is not about actual training so much as logistics.

Yes there are other reasons people cite for not wanting their work to be used as training material, and again I believe they have every right to have that preference, but personally I don't mind my public output being used by LLMs any more than I mind large search providers like Google having it in their index. I might have felt differently if I relied on obtaining direct Web traffic for on-page advertising to pay, but I do not subscribe to any advertising network (any advertisements you might see near my work are not mine and do not benefit me at all and probably shouldn't be there) and I was never convinced that on-page advertising was a sustainable model in all but a few cases anyway.

So happy reading LLMs (I copied much of my site to Markdown in my public Git repositories to make it easier) and I hope you manage genuinely to help somebody somewhere. Just try to guide them as ethically as you can. Please. Thank you.