Eli Lilly redux: Thinking about the democratization of information and the removal of gatekeepers alongside this story I saw first at Boys’ Club. This Harvard PhD is vibe-manufacturing schizophrenia drugs in his garage. Are we vibe-coding medicine?
AI in Practice
TechCrunch reports on the problems inherent to AI-generated restaurant advertising - it looks wrong.
Guerrilla marketing: When someone built a fake deodorant company with a lightweight marketing plan to capture long-tail search results, she found her way into AI recommendation results, showing how brands can gain visibility in emerging discovery systems.
To read: this report from Common Sense Media, “Talk, Trust and Trade-Offs: How and Why Teens Use AI Companions.” About a third of teens have turned to an AI for serious conversations about their own mental health and relationships, showing these tools are already embedded in teen social life.
Computer Love
I listened to two great podcasts this week exploring the tangle of human/AI relationships. In “Computer Love,” the guests explore whether these relationships are good for us, and in “Your AI Companion Doesn’t Love You,” they explore the technical and business implications of these relationships. Both of these feature podcaster and writer Bridget Todd, who is promoting her new book “Love at First Prompt.” As of today, about 1 in 5 young people in the U.S. have asked a chatbot for mental health advice, and 1 in 5 high schoolers have had a romantic AI relationship.
Note: Todd’s (excellent) podcast title is a reference to Rule 30 of the internet.
Chester argues that universities should govern AI by classifying data by sensitivity rather than approving specific tools. Public data goes anywhere –> regulated data requires an institutional contract –> sensitive data requires a local system. He also makes a renewed case for MS Copilot.
Gruber argues that AI text watermarking likely degrades prose quality and, even if it doesn’t, remains objectionable because it operates as an embargoed, unverifiable tag or signal controlled by providers like Anthropic and Google.
A person was found to be hiding an AI prompt injection in a legal filing that tells any AI that reads it to side with his perspective. The judge in this case likened it to “an automated agent communicat[ing] covertly with a juror during trial.”
Computah: Make it a link
I spent a little time with the blog last night and pulled together two new site features using Claude Cowork. The last time I experimented significantly with Claude like this was to use Claude chat to build the link log from scratch, walking it through my thinking in plain language, then copying and pasting its suggestions into the backend of the site and hitting publish. This time I used Cowork, the tool that runs in the browser, and it clicked through the screens itself, fully taking on the execution of tasks. I have some coding skills, but not the kind these changes required. If I were taking this on, I would need YouTube, Hugo for Dummies, and my own personal IT guy, and still probably couldn’t pull it together.
Last night I asked a few things of Claude: I asked Claude Cowork to get into the backend of the micro.blog site and change my theme to link each line in the linklog back to my original post. The date field in the right column now links back to the original post for each link logged. No problem, easy request with easy execution. I made the request, confirmed the plan, and went about my business while Claude made the edits.
I also asked it to analyze my post content and suggest category tags for groups of content, then to label that content correctly in the backend. Claude reviewed about 500 posts and suggested I add a few new categories to my blog: AI in Practice, Books & Reading and Writing & Language. Then I set up some auto-filters to run at publish time to automatically categorize posts based on keywords moving forward.
Finally, I manually added the archive page, which lets you sort posts by category or year. This means I now have a functional archive here. Enjoy my anodyne thoughts, dear reader.
Adjacent to my day job, I’ve been toying with Claude Pro now for about a year; in my experience, it has improved significantly within the last six months. It’s not perfect: my requested edits were completed, but it also changed the CSS on the linklog so some of the text is too light to read, which I didn’t ask for and don’t want. But it has arguably extended my ability to execute on work that requires skills I don’t otherwise have (such as design and coding). What I do bring to the table is an expansive practical background in publishing and production, and all the language to describe it.
Gender, Power and AI: Wrestling for the soul of the network, again
Stanford’s Clayman Institute ran a virtual panel this morning called “Gender, Power, and Artificial Intelligence,” with Safiya Noble (UCLA), Catherine D’Ignazio (MIT), Angèle Christin (Stanford), and moderator Genevieve Smith, a Clayman Institute Postdoctoral Fellow. The panel applied principles from feminist tech studies to the current moment, and covered how gender norms get encoded in data and reproduced by AI systems, and discussed whether the technology has real capacity for equitable design and implementation at scale.
Noble’s argument throughout is that the governance conversation has gotten too high-level and universalizing while the actual outputs of these systems have profound day-to-day consequences for specific people today. She named the role of AI in the recent gerrymandering of Louisiana and Indiana as examples, and called for tripling down on long-term social science research about AI’s impacts. She also pointed out that philanthropy is retreating from feminist academic and organizational work because that work originates from the same dynamics that critique philanthropy itself, precisely at a point when this research is sorely needed. A lot of money is moving in AI, and very little of it is funding the people best positioned to study how it impacts everyone downstream.
D’Ignazio was asked directly whether feminist generative AI at scale is possible. Her answer was no, with caveats, given who owns the technology today and the current emphasis on profit motive. She suggested it is more important to consider how to organize around our relationship to technology, and how we might approach questions of profit and ownership, policy and decision-making, and data and tech governance.
She provided an example of a reasonable use case by walking us through a project from her Data + Feminism Lab. The example is documented at length in her recent book “Counting Feminicide: Data Feminism in Action,” where her team partnered with activists who scour news reports to document the gender-related killing of women and girls, including cisgender and transgender women. The lab built a very lightweight AI-based approach that streamlines the scanning and identification of news stories as possible cases to include in their project, supercharging their work (note: very similar to how the NYT uses AI to analyze data for reporting). In this example, the AI’s job is task-scoped, democratically co-determined with the people who use it, and small. Smith picked this up: there is an idea baked into the current LLM moment that AI must scale to make it marketable, and the alternative is using purpose-built models that are right-sized against a body of work.
Christin spoke at length about how embodiment is one of the primary focuses of feminist theory, and how AI perpetuates the “disembodied” illusion of technology, and how this dynamic shows up in everything from the marketing to UX to user comprehension. This spoke to my thoughts on how the single-interface design of LLM chat reproduces Haraway’s “god trick,” knowledge that presents as universal while concealing the specific and situated position it comes from.
The parallel I kept returning to, listening to this, is one I think about often with my own cohort of early bloggers, women who grew up alongside the rise of the internet — and then the rise of ad tech. The internet of the late 1990s and early 2000s was being shaped by several camps: writers, students, information architects, and user-centric researchers who saw it as an information access network and a space of possibility; entrepreneurs and opportunists who saw it as a channel for marketing, monetization and extraction; and a smaller boycott camp that wanted to limit and refuse the whole personal computing and digital revolution altogether.
It was generally considered weird to be a girl on a computer or a woman on the internet — so weird that many of our peers didn’t recognize us at all — and we were there anyway, making stuff, witnessing, learning, advocating, producing, influencing. So when I watch some of my old peers, many of whom are professional writers and academics today, treat LLMs as a question of refusal rather than a condition to engage with critically, I worry we are abdicating a responsibility at precisely the moment when our technical and rhetorical expertise applies. Their refusal has good logic: user-centric researchers and communities engaged extensively with the early internet and the extractive camp won anyway, so why expect a different outcome here?
But Noble’s work on algorithmic bias attributes that failure not to engagement, but to the institutional and financial disadvantages that user-centric approaches operated under relative to gargantuan commercial interests. David and Goliath. That gap does not close through abstention. Understanding the trade-offs around tech, producing knowledge and analysis that does not depend on investors and marketers to frame the platform and the questions, requires presence. Refusal cedes so much ground.
Overall, the recommendations from the panel were practical. Noble called for people with capital (and the political will to spend it) to consider how to put money toward socially responsible research and development. D’Ignazio called for alternative funding infrastructure outside of venture capital logic, and pointed at European digital sovereignty models as worthy of consideration here. She also gestured at the popular AI Skeptics reading group as one current example of mad-and-commiserating-as-organizing that is creating safe psychological space for people to talk about AI and its tradeoffs. Christin’s recommendation was community organizing, on the grounds that LLMs are unpopular with a lot of people who feel there is no space to say so, and that finding those spaces is itself worthy because it provides shared language and awareness of others’ knowledge and experiences.
Personally, it was refreshing to hear reflections on the work (and the feelings) of being inside institutions that are being reshaped by AI, and being responsible for some of how that reshaping gets communicated and absorbed. I’m thinking about the incredible value of interdisciplinary governance, and how the commitment to governance is a specific position, and all the margins to consider.
Further reading:
Catherine D’Ignazio and Lauren Klein, Data Feminism. The foundational text on applying intersectional feminist thinking to data science practice.
Catherine D’Ignazio, Counting Feminicide: Data Feminism in Action. Extended case study of the grassroots data activism project D’Ignazio described on the panel.
D’Ignazio et al., “Feminicide and Counterdata Production.” Research paper on the counterdata methodology behind the femicide tracking project.
D’Ignazio et al., “Data Feminism for AI.” Conference paper extending the data feminism framework to questions specific to AI systems.
Safiya Noble, Algorithms of Oppression. Noble’s study of how commercial search engines reinforce racism and sexism through their ranking systems.
Donna Haraway, “Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective” (1988). The original essay where Haraway introduces the god trick and the case for situated, embodied knowledge against the view from nowhere.
Reflections on teaching fiction writing in the age of AI, from a professor with ten years of classroom experience teaching writing at MIT.
A new study suggests that people who use AI for writing are more able to detect AI writing than automated scanner tools. My current LLM pet peeve is how they use language like load-bearing, structural and legible to describe most ideas.
Adventures in AI: I asked a Claude agent (new Opus, Pro plan) to build a Google Doc template with multiple tabs, using an existing doc as reference. It failed three times over two days, burned thru tokens, never worked with Drive. Eventually it spat out text for me to paste into a doc I made myself.
Fellow Madisonians, someone pulled together a website ranking local businesses in Madison by how local they are (by what criteria, idk). In my experience, this is one way we’re likely to see AI used in the next couple of years, via prototyping and/or executing ideas that result in dynamic websites.
Centaurs and Cyborgs on the Jagged Frontier by Ethan Mollick in 2023: “On some tasks AI is immensely powerful, and on others it fails completely or subtly. And, unless you use AI a lot, you won’t know which is which.”
Anecdotally, I’ve seen two family court cases where one party submitted full AI chats — prompts and colorful complaints included — as formal filings. The complaints wouldn’t pass muster with a real lawyer, but the conflict was nurtured by AI nonetheless. One was dinged for wasting the judge’s time.
I’ve posted a couple of times about instances I’m aware of where people are using AI in pro se court cases, especially family courts. A new study shows evidence of increasing numbers in pro se cases at the federal level, exacerbating existing bottlenecks. Many trade-offs abound here.
A professor asked students to self-report AI usage on their homework, leading to lots of confusion and uproar. Points aside, it’s clear people want more clarity up front about when and whether to use LLM tools. In the meantime, treating students like they’re guilty until proven innocent is a bad MO.
Timothy Chester offers some thoughts on the place of AI-assisted software development in a modern research university, and suggests that just because you can doesn’t necessarily mean you should.
I’m following a guy in TX who is using AI to write and illustrate children’s books whole cloth, then self-publishes using Amazon, and getting recognition in his region as a laudable children’s author. The books are categorically not good. It’s like people are rewarding his content strategy.
Silicon sampling is the practice of using LLMs to run surveys without talking to any people at all.
Innovations in scamming: Folks are predicting that AI will supercharge scams alongside any technical and administrative innovation. Here’s one example of an unethical use of AI, where an internet-based GLP-1 hub used AI to generate fake product images and before and after photos of smiling patients.
Testing a new feature I created using a mix of open source code and Claude, hoping I didn’t break my own site. I pulled together a dynamic link library using a Hugo partial and some shortcode that automatically catalogs all of my outbound links into sortable lists.
Author Margaret Atwood plays with Claude and reports back on her experience.
Interesting read: NYT is using a custom LLM tool to track trends within the “manosphere,” as reported by the Nieman Journalism Lab.
A friend of the blog told me a story about a Substacker who uses AI to summarize books and then publishes AI-generated content about those summaries, never reading the books herself, and yet has a ton of followers. I’d guess at least some of those are purchased, betting that a high follower count will beget more followers by suggesting clout and credibility she didn’t earn as a reader talking to fellow readers. And followers aren’t subscribers, but that’s the business bet.
People are lookie-loos, they get curious when something is doing numbers and creating activity, so inflating follower counts is a real and persistent strategy. None of this is new. But best practices still hold regardless of which technologies you layer on top. Marketing erodes trust when it prioritizes short-term gains over honesty and reliability.
It’s strange to live in a time when you can’t reliably distinguish someone who has engaged with ideas from someone who automated the appearance of engaging with them.
Anecdotally hearing about LLMs being weaponized in divorce and custody, including inundating the other party with slop to drive up the opponent’s legal fees. Worse, the sycophancy is tuned to and confirms the aggrieved party’s grievances, regardless of their real-world relevance in court.
Through a new quiz, NYT asks readers to rate passages of writing against AI. Despite thinking I could spot the AI writing, my results were 50/50.
AI in practice: Chatbot tool comparison
Come look over my shoulder while I explore how and whether LLMs are good writing tools: Here’s a wee version of the LLM comparison exercise I did with my team. We’ll make it a two-fer so you can see how the “good writing” skill works in practice, though we’ll see how that actually goes.
One of the more useful things you can do with an LLM is hold up a few ideas side by side and apply lenses to them. I know this history pretty well, so I asked a series of LLMs, why is Wisconsin’s cultural identity and cohesion stronger than Indiana’s, from a historical and business perspective?
Here are the answers in one doc, for comparison.
Each LLM will give us more or less the same story, different flavor. Within the industry, the differences across the models reflect “model personality.” Asking “why” instead of “whether” will probably drive the answer to favor Wisconsin. Using multiple lenses (two states, historical + business, identity + cohesion) forces the LLM to cross-reference across more of its training data, which tends to produce a more comprehensive answer.
For all the chatter about consciousness and whatever, remember that an LLM is an infinite series of if/then/elses applied to human language and semantics, so being able to talk about language and communication, getting meta with the tool and how you think through language, helps a lot when using one. This is maybe the one thing I like about experimenting so hard with the tools. I’m thinking about the technical side of writing and enjoying it quite a lot.
Functionally: all of them acknowledge hard historical truths within the subject matter and don’t shy away from critical perspectives, which is good. Both Gemini and Copilot include in-line links, which lets you judge the output’s authority in the moment as a reader. I liked Copilot’s more than I expected here. Claude’s answers are more lyrical and do provide more context, and yet do not encourage checking against outside sources by providing links within the output. And you can see that even with the good writing skill calling out hard bans on certain structure, Claude plows right through them.
Model personality: Claude favors sociological answers to Copilot’s economic answers. Claude is also highly intellectual and narrative by comparison, and that narrative style can mask nuance by sinking relative context within the storytelling. Gemini simplifies, boosts and cheerleads where the others don’t, and really goes hard on Wisconsin’s reputation as a drinking and Packers state when there are stronger structural arguments in play. Copilot is tricky because it looks authoritative like a briefing, which also makes it easily “extractible” for the user, but every citation requires authentication unless this is one of those “good enough” tasks.
As a writer, something I find annoying across the whole spread is the semantic reveal. LLMs are semantic machines, and it is persistently revealed in ways that are weird to the human ear. All of them go out of their way to describe things as “structural,” “connective” as in “connective tissue,” “load-bearing” and “legible.”
Finally, I included a second tab where I asked Claude for analysis across the four outputs, where it suggests that my framing of the question is altogether kind of problematic. It shows how a strong prompt is sometimes also a bad approach.
There are a lot of possible takeaways here, but I’d rather set aside the question of which tool is “good” or “bad” or “better” and think more about the patterns across the tools and their implications.
One of the tricky things about consumer AI tools like Claude and Gemini is that the experience varies widely depending on the person using it, and it’s not always clear why. I have spent a lot of time learning the tools so I can advise on them in my work, and this variance of experience has become a frustrating part of the deal.
I manage a team of writers and creatives at work, and we are expected to be familiar with the tools, despite complex and sometimes hostile feelings about the political and environmental implications of this sector. That’s quite a pickle, organizationally, managerially. Borrowing from Haraway, I thought, okay, what if we take these tools seriously as a team of writers and creatives and put our professional standards up against them?
Among other exercises, I did a couple of comparisons on my team that help create discussion around the “plausibility” question. People dismiss LLM outputs as being merely plausible answers, rather than accurate or factual ones. And that’s correct; they are, and that’s the design. In many cases, plausibility is fine. Take Wikipedia, for example, which we understand to be a pretty good source, a plausible source, unless you’re writing a formal paper requiring original sources.
I digress. Ultimately, we needed to understand together that LLMs are not a WYSIWYG tool and talk through the implications.
I asked everyone to run the same paper through their LLM of choice, prompting it for a plain language summary. We then copied and pasted it into a shared doc, and compared and contrasted for discussion. Upon discussion, we had several takeaways, including that they were all similar in spirit but sometimes varying wildly in style and approach.
Knowing that algorithms are responsive and not static, we did it again later in the day and copied and pasted our outputs into the shared doc. We compared and contrasted the difference between AM and PM. Again, it was similar in spirit but varied in style and approach. Some changed dramatically. One team member whose morning summary had been jokey and conversational received a much more staid and serious version in the afternoon.
At the time, I asked Claude to explain the variance: “Even with the same prompt and source material, LLMs don’t produce identical outputs each time. This is by design — there’s a degree of randomness (called “temperature”) in how the model selects words, which means each run produces a slightly different path through the text.”
Anyway, this got our gears turning on how (and whether) to approach LLMs as a team and as individuals and led to good group discussion. (It’s important to create space for criticism and critical approaches here.) It also gave us more confidence as a team responding to this new layer of complexity in our work, and helping our professional contacts and peers think about how to approach the tools and when and whether to use them. There will be tasks where AI-based tools are “good enough,” and tasks where they are not.
The swirl of mystery and speculation around this sector has people up in arms, and it’s useful to have approaches that give people firsthand experience and to see how the experience works for others. The god trick of the singular interface turns out to be a bear for navigating it in the workplace, where our work is foundational, prosocial and specific.