theluddite

theluddite , 4 days ago

This is bad science at a very fundamental level.

Concretely, we deploy GPT-4 as an agent in a realistic, simulated environment, where it assumes the role of an autonomous stock trading agent. Within this environment, the model obtains an insider tip about a lucrative stock trade and acts upon it despite knowing that insider trading is disapproved of by company management.

I’ve written about basically this before, but what this study actually did is that the researchers collapsed an extremely complex human situation into generating some text, and then reinterpreted the LLM’s generated text as the LLM having taken an action in the real world, which is a ridiculous thing to do, because we know how LLMs work. They have no will. They are not AIs. It doesn’t obtain tips or act upon them – it generates text based on previous text. That’s it. There’s no need to put a black box around it and treat it like it’s human while at the same time condensing human tasks into a game that LLMs can play and then pretending like those two things can reasonably coexist as concepts.

To our knowledge, this is the first demonstration of Large Language Models trained to be helpful, harmless, and honest, strategically deceiving their users in a realistic situation without direct instructions or training for deception.

Part of being a good scientist is studying things that mean something. There’s no formula for that. You can do a rigorous and very serious experiment figuring out how may cotton balls the average person can shove up their ass. As far as I know, you’d be the first person to study that, but it’s a stupid thing to study.

theluddite , 4 days ago (edited 4 days ago)

You can’t use an LLM this way in the real world. It’s not possible to make an LLM trade stocks by itself. Real human beings need to be involved. Stock brokers have to do mandatory regulatory trainings, and get licenses and fill out forms, and incorporate businesses, and get insurance, and do a bunch of human shit. There is no code you could write that would get ChatGPT liability insurance. All that is just the stock trading – we haven’t even discussed how an LLM would receive insider trading tips on its own. How would that even happen?

If you were to do this in the real world, you’d need a human being to set up a ton of stuff. That person is responsible for making sure it follows the rules, just like they are for any other computer system.

On top of that, you don’t need to do this research to understand that you should not let LLMs make decisions like this. You wouldn’t even let low-level employees make decisions like this! Like I said, we know how LLMs work, and that’s enough. For example, you don’t need to do an experiment to decide if flipping coins is a good way to determine whether or not you should give someone healthcare, because the coin-flipping mechanism is well understood, and the mechanism by which it works is not suitable to healthcare decisions. LLMs are more complicated than coin flips, but we still understand the underlying mechanism well enough to know that this isn’t a proper use for it.

theluddite , 4 days ago

Regardless of their conclusions, their methodology is still fundamentally flawed. If the coin-flipping experiment concluded that coin flips are a bad way to make health care decisions, it would still be bad science, even if that’s the right answer.

theluddite , 4 days ago

Thanks! There are tons of these studies, and they all drive me nuts because they’re just ontologically flawed. Reading them makes me understand why my school forced me to take philosophy and STS classes when I got my science degree.

theluddite , 3 days ago

Haha no that's not complaining; it's good feedback! I've been meaning to do that for a while but I'll bump it up my priorities.

theluddite , 16 days ago

If you don’t already know about it, I think you’ll like low tech magazine. It’s basically an entire website saying what you just said for over ten years now.

solar.lowtechmagazine.com

theluddite , 16 days ago

My favorites, though it’s really hard to choose:

…lowtechmagazine.com/…/fruit-trenches-cultivating…

…lowtechmagazine.com/…/fruit-walls-urban-farming-…

…lowtechmagazine.com/…/restoring-the-old-way-of-w…

theluddite , 22 days ago (edited 22 days ago)

At some point in the last decade, the ostensive ostensible goal of automation evolved from savings us from unwanted labor to keeping us from ever doing anything.

theluddite , 22 days ago

Yeah I agree. That’s why I said it was their ostensive goal. Their actual goal has only ever been profit.

theluddite , 22 days ago

Oh huh TIL. I also looked it up, and it seems like a real doozy of a word. I had no idea. Looks some some dictionaries say that the two words are interchangeable, whereas others distinguish between them, and in the latter case, I used the wrong one. Language is fun!

theluddite , 22 days ago

Not if you’re too busy between your two jobs manually training the LLM models and supervising the supposedly autonomous cars to make rent.

theluddite , 24 days ago

The goal of every company is to do shenanigans at the top while profiting off an underclass of laborers at the bottom. The more shenanigans they can do to squeeze the underclass harder, the better. Uber et al are genuine innovators in automating labor law violations to maximize that squeeze. Looks like they’re expanding from chauffeurs to every other kind of household servant. Awesome. This will be very cool and fine.

theluddite , 27 days ago

Our entire news ecosystem is putrid trash. Even our most prestigious and respected outlets are pumping out a constant stream of genocide apologia right now. Manufacturing Consent is decades old and should’ve ended the New York Times, and that was before they cheerlead our war into Iraq.

Allowing advertising to decide which content is allowed and which isn’t won’t do anything but punish sites that deviate from mainstream orthodoxy and reward bland corporate friendly bullshit. Here’s what that Internet looks like.

theluddite , 28 days ago

Yup. It’s a technological antisolution.

theluddite , 1 month ago

You’re not wrong but I think you’re making too strong a version of your argument. Many people, including wealthy people, are genuinely, deeply moved by art. I love the symphony, opera, and ballet. If I were rich I’d absolutely commission the shit out of some music and get a lot of real joy out of that.

theluddite , 1 month ago

I do software consulting for a living. A lot of my practice is small organizations hiring me because their entire tech stack is a bunch of shortcuts taped together into one giant teetering monument to moving as fast as possible, and they managed to do all of that while still having to write every line of code.

In 3-4 years, I’m going to be hearing from clients about how they hired an undergrad who was really into AI to do the core of their codebase and everyone is afraid to even log into the server because the slightest breeze might collapse the entire thing.

LLM coding is going to be like every other industrial automation process in our society. We can now make a shittier thing way faster, without thinking of the consequences.

theluddite , 1 month ago

Yeah, I think helping people who don’t know how to code and letting them dabble is a great use case. I fully encourage that.

I don’t think it’s actually good for generating scaffolding in terms of helping people write quality software, but I do agree with you that that’s how people are going to use it, and then the expectation is going to become that you have to do things that fast. It’s kind of mindboggling to me that anyone would look at the software industry and decide that our problem is that we don’t move fast enough. Moving too fast for speed’s own sake is already the cause of so many of our problems.

theluddite , 1 month ago

Yeah, I totally see that. I want to clarify: It’s not that I don’t think it’s useful at all. It’s that our industry has fully internalized venture capital’s value system and they’re going to use this new tool to slam on the gas as hard as they can, because that’s all we ever do. Every single software ecosystem is built around as fast as possible, everything else be damned.

theluddite , 29 days ago

I agree. I’ve actually written about this.

It gets solved by planning. Actual long term planning that includes the relevant stakeholders. Currently everything is run by and for VCs who only plan in terms of funding rounds and exits.

theluddite , 29 days ago

Honestly I almost never have to deal with any of those things, because there’s always a more fundamental problem. Engineering as a discipline exists to solve problems, but most of these companies have no mechanism to sit down and articulated what problems they are trying to solve at a very fundamental level, and then really break them down and talk about them. The vast majority of architecture decisions in software get made by someone thinking something like “I want to use this new ops tool” or “well everyone uses react so that’s what I’ll use.”

My running joke is that every client has figured out a new, computationally expensive way to generate a series of forms. Most of my job is just stripping everything out. I’ve replaced so many extremely complex, multi-service deploy pipelines with 18 lines of bash, or reduced AWS budgets by one sometimes two orders of magnitude. I’ve had clients go from spending 1500/month on AWS with serverless and lambda and whatever other alphabet soup of bullshit services that make no sense to 20 fucking dollars.

It’s just mind-blowing how stupid our industry is. Everyone always thinks I’m sort of genius performance engineer for knowing bash and replacing their entire front-end react framework repo that builds to several GB with server side templating from 2011 that loads a 45kb page. Suddenly people on mobile can actually use the site! Incredible! Turns out your series of forms doesn’t need several million lines of javascript.

I don’t do this kind of work as much anymore, but up until about a year ago, it was my bread and butter…

theluddite , 1 month ago

I do a lot of writing of various kinds, and I could not disagree more strongly. Writing is a part of thinking. Thoughts are fuzzy, interconnected, nebulous things, impossible to communicate in their entirety. When you write, the real labor is converting that murky thought-stuff into something precise. It’s not uncommon in writing to have an idea all at once that takes many hours and thousands of words to communicate. How is an LLM supposed to help you with that? The LLM doesn’t know what’s in your head; using it is diluting your thought with statistically generated bullshit. If what you’re trying to communicate can withstand being diluted like that without losing value, then whatever it is probably isn’t meaningfully worth reading. If you use LLMs to help you write stuff, you are wasting everyone else’s time.

theluddite , 1 month ago

Amen. In fact, I wrote a whole thing about exactly this – without an LLM! Like most things I write, it took me many hours and evolved many times, but I take pleasure in communicating something to the reader, in the same way that I take pleasure in learning interesting things reading other people’s writing.

theluddite , 1 month ago

I don’t think that sounds like a good way to make a good paper that effectively communicates something complex, for the reasons in my previous comment.

theluddite , 1 month ago

Top-down bureaucracies are bad at adoption. That’s just obvious at this point. If you want to use a computer to fix this problem, you can’t simply automate the existing structure. You need to actually think about how you can use the computer to do something qualitatively and structurally different than what we’re currently doing, instead of the same basic thing but faster and with more data.

This is why I say that capitalism uses computers backwards. I even used online dating as an example when I wrote that almost a year ago. If you think within capitalism, and you incorporate yourself as a capitalist firm, even if you try to do good things, the structure of your solution will reflect that of your organization, and many of our problems simply don’t respond well to that.

theluddite , 1 month ago

If you take it as a given that we should have giant warehouses full of computers using tons of energy while doing mostly pointless tasks during a climate emergency, then yes, it’s a great idea.

theluddite , 1 month ago

That’s a bad faith gotcha and you know it. My lemmy account, the comment I just wrote, and the entire internet you and I care about and interact with are a tiny sliver of these data warehouses. I have actually done sysadmin and devops for giant e-commerce company, and we spent the vast majority of our compute power on analytics for user tracking and advertising. The actual site itself was tiny compared to our surveillance-value-extraction work. That was a major e-commerce website you’ve heard of.

Bitcoin alone used half a percent of the entire world’s electricity consumption a couple of years ago. That’s just bitcoin, not even including the other crypto. Now with the AI hype, companies are building even more of these warehouses to train LLMs.

theluddite , 1 month ago (edited 1 month ago)

I didn’t say there are no good uses for data. Of course there are! I even wrote “useless things” in the comment to distinguish from real uses.

Personally I think self driving cars are never going to happen and the LLM coding hype fundamentally misunderstands what software does and is actually for, but even though I don’t agree with your examples, only a complete fucking moron would think computing in general is useless. My point is that current computing practices are insanely wasteful.

theluddite , 1 month ago

From that same article stub:

The nonprofit DataKind has a partnership with John Jay College of Criminal Justice, where 44% of students are Hispanic, to run a predictive AI program that helps identify students — especially those from low-income families — who are in danger of dropping out because of grades or other factors.

This is a very dangerous path. I recognize it thanks to Dan Mcquillan, who writes about this a lot. Governments using algorithmic tools to figure out who needs special services ends up becoming automated neoliberal austerity. He frequently collects examples. I just dug up his mastodon and here’s a recent toot with three: kolektiva.social/…/111207202749078945

Also, the main headline is about automated text translations for calls, which is now AI. Ever since ChatGPT melted reporters’ brains, everything has become AI. Every time I bring this up, some pedantic person tells me that NLP (or machine vision or LLMs) is a subfield of AI. Do you do this for any other field? “Doctors use biology to solve disease,” or “Engineers use physics to to build bridge.” Of course not, because it’s ridiculous marketing talk that journalists should stop repeating.

theluddite , 1 month ago

Copyright is broken, but that’s not an argument to let these companies do whatever they want. They’re functionally arguing that copyright should remain broken but also they should be exempt. That’s the worst of both worlds.

theluddite , 1 month ago

Computers aren’t people. AI “learning” is a metaphorical usage of that word. Human learning is a complex mystery we’ve barely begun to understand, whereas we know exactly what these computer systems are doing; though we use the word “learning” for both, it is a fundamentally different process. Conflating the two is fine for normal conversation, but for technical questions like this, it’s silly.

It’s perfectly consistent to decide that computers “learning” breaks the rules but human learning doesn’t, because they’re different things. Computer “learning” is a a new thing, and it’s a lot more like creating replicas than human learning is. I think we should treat it as such.

theluddite , 1 month ago

Yes 100%. Once you drop the false equivalence, the argument boils down to X does Y and therefore Z should be able to do Y, which is obviously not true, because sometimes we need different rules for different things.

theluddite , 1 month ago

😘

theluddite , 1 month ago

It’s not that this article is bad, but it is what frustrates me about tech journalism, and why I started writing about tech. None of these people have any idea how the internet actually works. They’ve never written a line of code, or set up a server, or published an app, or even done SEO, so they end up turning everything into a human interest piece, where they interview the people involved and some experts, but report it with that famous “view from nowhere.”

Some blame Google itself, asserting that an all-powerful, all-seeing, trillion-dollar corporation with a 90 percent market share for online search is corrupting our access to the truth. But others blame the people I wanted to see in Florida, the ones who engage in the mysterious art of search engine optimization, or SEO.

Let me answer that definitively: it’s google, in multiple ways, one of which isn’t even search, which I know because I actually do make things on the internet. SEO people aren’t helping, for sure, but I’ve seen many journalists and others talk about how blogspam is the result of SEO, and maybe that’s the origin story, but at this point, it is actually the result of google’s monopoly on advertising, not search. I’ve posted this before on this community, but google forces you to turn your website into blogspam in order to monetize it. Cluttering the internet with bullshit content is their explicit content policy. It’s actually very direct and straightforward. It’s widely and openly discussed on internet forums about monetizing websites.

theluddite , 1 month ago

What client are you on? There’s a known bug in some lemmy clients that break some URLs.

If you paste the URL into a browser it should work fine: theluddite.org/#

theluddite , 1 month ago

Hey thanks so much friend. You should submit a hall of shame entry! We rarely get submissions and I agree it’s such a fun part of the site.

theluddite , 1 month ago

I was in tears of laughter while making it. I couldn’t believe when they accepted it except part of me always totally expected it because they’re fucking clowns.

theluddite , 1 month ago

lmao thank you. That’s slightly strange but extremely nice to read.

theluddite , 1 month ago

That is only true if you use capitalist metrics to measure poverty

(1) It is unlikely that 90% of the human population lived in extreme poverty prior to the 19th century. Historically, unskilled urban labourers in all regions tended to have wages high enough to support a family of four above the poverty line by working 250 days or 12 months a year, except during periods of severe social dislocation, such as famines, wars, and institutionalized dispossession – particularly under colonialism. (2) The rise of capitalism caused a dramatic deterioration of human welfare. In all regions studied here, incorporation into the capitalist world-system was associated with a decline in wages to below subsistence, a deterioration in human stature, and an upturn in premature mortality. In parts of South Asia, sub-Saharan Africa, and Latin America, key welfare metrics have still not recovered. (3) Where progress has occurred, significant improvements in human welfare began several centuries after the rise of capitalism.

theluddite , 1 month ago

Paul Krugman is an innovator in this field. The other day he had that one about how inflation is under control if you remove, food, energy, used cars, and everything else normal people use. That’s basically all my stuff!

Also, obligatory GDP joke that’s been bouncing around the internet for a while now:

As they’re walking, they come across a pile of dog shit. One economist says to the other, “If you eat that dog shit, I’ll give you $50”. The second economist thinks for a minute, then reaches down, picks up the shit, and eats it. The first economist gives him a $50 bill and they keep going on their walk. A few minutes later, they come across another pile of dog shit. This time, the second economist says to the first, “Hey, if you eat that, I’ll give you $50.” So, of course, the first economist picks up the shit, eats it, and gets $50. Walking a little while farther, the first economist looks at the second and says, “You know, I gave you $50 to eat dog shit, then you gave me back the same $50 to eat dog shit. I can’t help but feel like we both just ate dog shit for nothing.” “That’s not true”, responded the second economist. “We increased the GDP by $100!”

theluddite , 2 months ago

I am totally in favor of criticizing researchers for doing science that actually serves corporate interests. I wrote a whole thing doing that just last week. I actually fully agree with the main point made by the researchers here, that people in fields like machine vision are often unwilling to grapple with the real-word impacts of their work, but I think complaining that they use the word “object” for humans is distracting, and a bit of a misfire. “Object detection” is just the term of art for recognizing anything, humans included, and of course humans are the object that interests us most. It’s a bit like complaining that I objectified humans by calling them a “thing” when I included humans in “anything” in my previous sentence.

Again, I fully agree with much of their main thesis. This is a really important point:

As co-author Luca Soldaini said on a call with 404 Media, even in the seemingly benign context of computer vision enabled cameras on self-driving cars, which are ostensibly there to detect and prevent collision with human beings, computer vision is often eventually used for surveillance.

“The way I see it is that even benign applications like that, because data that involves humans is collected by an automatic car, even if you’re doing this for object detection, you’re gonna have images of humans, of pedestrians, or people inside the car—in practice collecting data from folks without their consent.” Soldaini said.

Soldaini also pointed to instances when this data was eventually used for surveillance, like police requesting self-driving car footage for video evidence.

And I do agree that sometimes, it’s wise to update our language to be more respectful, but I’m not convinced that in this instance it’s the smoking gun they’re portraying it as. The structures that make this technology evil here are very well understood, and they matter much more than the fairly banal language we’re using to describe the tech.

theluddite OP , 2 months ago

I post our stuff on lemmy because I’m an active user of lemmy and I like it here. I find posting here is more likely to lead to real discussions, as opposed to say Twitter, which sucks, but is where I’d be if I was blasting self-promotion. It’s not like lemmy communities drive major traffic.

Isn’t that exactly what lemmy is for? It’s what I used to love about Reddit 10 years ago, or Stumble Upon, or Digg, or any of the even older internet aggregators and forums: People would put their small, independent stuff on it. It’s what got me into the internet. I used to go on forums and aggregators to read interesting stuff, or see cool projects, or find weird webcomics, or play strange niche web games, or be traumatized by fucked up memes. Now the entire internet is just “5 big websites, each consisting of pics from the other 4” or whatever the quip is, and it’s fucking boring.

So yes, I and a few others are theluddite.org. It’s an independent site written by leftists working in tech and academia, mostly aimed at other people in tech and academia, but also for everyone. It’s not like I’m hiding it; it literally says so in my bio. We are not professional opinion-havers, unlike “mainstream” sources; I personally write code for a living every day, which is something that surprisingly few tech commentators have ever done. That makes it possible for me to write about major topics discussed in the media, like google’s ad monopoly,, in a firsthand way that doesn’t really exist elsewhere, even on topics as well trodden as that one.

And yes, we post our stuff on the fediverse, because the fediverse rules. It is how we think the internet should be. We are also self-hosted, publish an RSS feed, don’t run any ads or tracking (and often write about how bad those things are for the internet) because that’s also how we think the internet is supposed to work.

theluddite , 2 months ago

There are two issues. First, self-driving cars just aren’t very good (yet?). Second, it will make millions of people’s jobs obsolete, and that should be a good thing, but it’s a bad thing, because we’ve structured our society such that it’s a bad thing if you lose your job. It’d be cool as hell if it were a good thing for the people who don’t have to work anymore, and we should structure our society that way instead.

theluddite , 2 months ago (edited 2 months ago)

I’m not sure if that article is just bad or playing some sort of 4D chess such that it sounds AI written to prove its point.

Either way, for a dive into a closely related topic, one that is obviously written by an actual human, I humbly submit my own case study on how Googles ad monopoly is directly responsible for ruining the Internet. I posted it here a week ago or so, but here it is in case you missed it and this post left you wanting.

theluddite , 2 months ago

Yeah but I can tell you if something is a crosswalk

theluddite OP , 3 months ago

Yeah absolutely. The Luddite had a guest write in and suggest that if anxiety is the self turned inwards,nthe internet is going to be full of increasingly anxious LLMs in a few years. I really liked that way of putting it.