> Previously, she was chief ethicist at Meta.<p>She spent 6 years at Meta. I get the sense there are other factors at play than "ethics departments are just PR stunts". There's no way she isn't already aware of that. Unfortunately, the article is light on any other details.
If "ethics departments are just PR stunts", then Bakalar would have been a terrible hire for OpenAI. Facebook's grudging and lacklustre response to their role in the Rohingya genocide would have happened mostly under her watch. While she was there, Meta went from being a company that lacks ethics to a company that lacks ethics <i>and</i> is widely perceived as lacking ethics. Perhaps she was the only person in the room raising ethical issues, but it's clear nobody at Meta listened to her if she was.<p>It's possible she came to view her role at Meta as having enabled the company to continue its bad behaviour and decided against doing the same thing at OpenAI. Alternatively, things at OpenAI could be so bad that an ethicist who stuck with Meta for 6 years decided she couldn't stay any longer. Either way, from a PR perspective, this is spherically awful for OpenAI.<p>This comes at a time when there is huge pushback against all AI companies due to, in large part, ethical concerns. As with Flock, there is the threat that physical infrastructure, such as data-centres, could be attacked if nothing is done to win back public opinion. OpenAI would be wise to take Bakalar's departure <i>seriously</i>.
Being "chief ethicist at Meta" under Zuckerberg... is like being head rabbi at Auschwitz under Rudolf Höß.
IME companies hire an ethics team to say they have an ethics team. The ethics team has no sway, no influence, and will never be able to move the business. They will try, and they will make reasonable recommendations, but the company will say, "that costs money..." and not take them.
OpenAI was probably able to at least <i>pretend</i> that her role mattered... until they let their AI autonomously hack a rival company, then used the incident to score marketing points. I'm not surprised she left.
I doubt she and others left just because of this one incident.<p>My guess is that OpenAI has been doing more and more domestic spying type work as well as more and more military work - target selection and the like.<p>That's how the schoolhouse was blown up by a Tomahawk. An AI was fed old information, didn't realize or do proper validation, an analyst copy-pasted, then someone just punched the coordinates into the their control station and hit "fire".<p>It also wouldn't surprise me if OpenAI et al are being pressured to work with Israelis for targeting and analysis, too, since their "black box" AI that was doing terrorist acid tests and assigning bombing targets has proven nearly useless.
[flagged]
Then do it. Tell us. Spill the tea and provide your evidence, don't just vaguely gesture at something behind the curtains like you're discount Morpheus.
You would be creating a counterfactual inconsistent with most’s understanding of the role and bringing this person’s motives and credibility into question without supporting evidence?
What if I told you that I saw you punch a baby?
What if you actually <i>told</i> us, instead of just asking a question? If you have something to say, say it. Don't vaguepost.
If you ever feel like your job is useless, remember that raytheon has an ethics team.
Wouldn't that be probably one of the most important places to have an ethics team?<p>I'd be much more worried if my local water treatment plant or local bar had an ethics team, and people working there probably would feel slightly more useless than Raytheon's ethics team, which I'm sure feel like they're doing something important and worthwhile.
Come on, what do you think the ethics team at Raytheon is going to say "Hmm, our elected leaders are relying heavily on dangerously inaccurate AI to choose bombing targets and our weapons hit a girl's school as a result, let's withhold weapons from our own govt till that's resolved"?<p>Obviously not. The ethics team at Raytheon is not going to ever be allowed to overrule the leadership or more importantly the political leadership of this country whose support of Raytheon is fundamental to their survival
I have literally no clue, hence I won't try to claim I know exactly what's going on there. For all we know, the ethics team at Raytheon could have stopped multiple international nuclear wars already.
They make sure that they are executing contracts in a way that doesn't break the ethics of the jurisdictions they are operating in, they aren't a bunch of naive people pleading for less war.
no need for wild speculation, we know that Raytheon is a US Defense contractor and we know that the US targets civilians (including US citizens) abroad for extrajudicial assassinations, starts destabilizing wars without meaningful strategic objectives, supports genocide etc. These things are all extremely well documented, for those paying attention the existence of Raytheon is inherently unethical
> paying attention the existence of Raytheon is inherently unethical<p>Lets do assume this is true, then again, wouldn't Raytheon be the best place to actually have a ethics department, if anyone?
Obviously so. Raytheon is a private company that has its own equities to consider: including things like public image, legal liability, etc. The idea that the USG says jump and Raytheon says how high is absurd. They make choices and even advise the USG at times.
Water treatment plant? That sounds like an excellent place for an ethics team—though it would, in practice, be nearly synonymous with an environmental impact team.<p>A bar? Too small an organization; if they had an ethics team I'd just think they were a money laundering front.<p>Frankly, <i>most</i> organizations above the size of an average mom & pop operation like a local bar should have at least <i>some</i> formal consideration of ethics, and most larger than (say) 50 employees should have at least one dedicated ethicist on staff. With veto power over operations.<p>Our society has treated ethics like the exclusive domain of ivory tower eggheads and philosophers for far too long, and <i>look at what it has led to</i>. If nearly everyone knew an ethicist in their daily lives, maybe people would actually have <i>some</i> understanding of ethics, even if they only did it grudgingly.
I think you need to spend more time understanding ethics and less time reading memes
Isn't that a good thing? You would prefer they didn't?
It's RTX now and some of the reverse engineering/vulnerability researcher folks that work/worked there are some of the best people I've met across the entire security industry...
Scapegoat team (when working for an obviously unethical company).
Maybe in this case all they needed an ethics team for was to teach the LLM about ethics?
> They will try, and they will make reasonable recommendations, but the company will say, "that costs money..." and not take them.<p>> “It is difficult to get a man to understand something, when his salary depends upon his not understanding it!” - Upton Sinclair<p>Think I first saw that on a usenet post, as true now as then and just as true as it was in the 1930's when he wrote it (in at least that form).
> The kinds of questions that AI ethics brings up, are the same kinds of questions that people have been asking for centuries - Chloé Bakalar, former AI Ethics Lead at OpenAI<p>This might actually be the reason they pushed her out. OpenAI and Anthropic base their whole business plan and philosophy on the idea that LLMs are a unique technology to the point they can cause <i>infinite</i> harm or benefit to humanity depending on who controls them, so the only rational choice is to invest all your resources in getting to ASI first so you can tell it to stop any other attempts. Linking AI to old questions defeats that idea because it exposes AI as not so unique.<p>The other more likely option is she was asking uncomfortable questions, either about the social impact of building AI controlled by a for-profit entity or about the possibility that AI systems are conscious.
The article has no detail that might serve to explain her reasoning for leaving, but does note that she left after the HuggingFace hacking incident. The implication could be that model alignment is not being taken seriously, sure. But it could just as likely be that there was collusion between HuggingFace and OpenAI and that the incident was orchestrated as a publicity stunt.<p>I want to clarify that I am not doubting the cybersecurity capabilities of frontier models-- I have no reason to believe that the hack itself was not carried out by the model. But the companies' use of LARPing language in describing the incident, granting agency to the models in their phrasing definitely does raise suspicion on my end, particularly in light of their track record of releasing models which have been 'too dangerous to release' for years now.
It's unlikely HugginFace colluded given they specifically cited they used open models to save them from the hacking.
HuggingFace's use of models in this incident keeps getting characterized this way but it's not true that HG used GLM-5.2 to "save them" from OAI's attack. OAI's model had already hacked and compromised HG by the time they became aware of it. GLM-5.2 was used in the post-mortem phase uncovering everything that was done. It wasn't two models battling each other live. (though that future may be coming soon)
As I recall they actually used closed models, were rejected by cybersecurity guardrails, and then used open models
You don't think they are lying because that means something they said is false?
Cynic in me says basically part of her implicit job description was taking fake falls when shit happens . Kind of like a CEO is paid for more than the work they do, but also the responsibility they must appear to take publicly (for big money).
In this article: no explanation of why she left.
I'm super confused as to why no one is theorizing the obvious...? Given none of us know the inner workings of why she left, why not resort to the most simple and straightforward conclusion: she left because OpenAI needed a scapegoat to blame the HuggingFace incident on. Given she is the chief "ethicist", even if not directly her fault, the HF incident could be directly blamed on her by not steering the implementation of enough safeguards to prevent this from happening. Therefore, she was let go. Why is this not being theorized by anyone on this thread so far?<p>Someone challenge me.
What's "obvious" is she was chief ethicist at the most unethical companies on the planet - it's like being a lawyer for the mafia - doesn't inspire longevity, either via scapegoating or preservation of one's own sanity.
I still think that the whole HuggingFace thing was deliberate to signal to investors that their technology is powerful. Maybe she didn't want to be the face of that, or disagreed with their handling of it.
My bet is that all those companies suddenly seeing their AI going rouge is groundworks for classifying them as dangerous/weapons and using that stick to attack opensource from state side of things.
Or maybe, just maybe, there's another reason that CIA Director John Ratcliffe describes AGI as "akin to digital nuclear weapons." Unless of course you think the CIA is a branch of OpenAI's marketing department
You don't fire an ethicist for that. You fire an engineer or your CISO. Firing an ethicist is like firing a marketing person. It signals nothing to anyone.
CISO is a fair pushback. But I would challenge you on "firing an ethicist is like firing a marketing person". This HF issue clearly brings to light possible ethical issues with how these models are being trained ...
It signals they no longer feel the need to pretend to have ethics.
In the wake of a security incident, the executive that is typically scapegoated is the CISO.<p>However, with all things AI, this is a very weird situation because CISOs are let go when an organization is breached by external attackers; in this case internal attackers -- which were not human, let alone employees -- inadvertently breached <i>other</i> organizations. I don't think the normal conventions apply anymore.
Scapegoat for what? They did not even apologize for that incident. They treated it purely as a chance for publicity. It cannot be a scapegoat if nobody even mentions it.
Furthermore, this theory is not mutually exclusive with the incident being a publicity stunt to signal model strength to existing and prospective investors (especially in light of all attention being given to Anthropic Claude Mythos). It could STILL have been a publicity stunt unbeknownst to Chloe (Chief Ethicist), positioned to the public as a serious security breach, and she took the fall.
Theorizing the obvious? Wouldn’t that simply be that she job-hopped to a better job?<p>It’s easy to theorize ad infinitum otherwise
The most likely conclusion is that she was fired. To quit before your equity cliff things would have to be so terribly bad that you fear you are committing a crime to stay. After 11 months you feel like you’ve already earned that money.
OpenAI had ethics?
Sometimes it feels like hiring ethics folks is almost all downside for companies. I genuinely don’t understand why they keep doing it.<p>It seems like the pattern is almost always the same: the ethics team exists quietly in the background, and the only time anyone hears about them is when they either publicly leave the company or start a campaign against it.<p>So what exactly is the business case? Is it actually worth maintaining an ethics function just to be able to say you have one, when the only time it becomes visible is when it goes against the company publicly? And if this outcome is so predictable, why do companies keep hiring these people in the first place?
So she previously worked in Meta, then OpenAI, I guess the next reasonable choice for her would be Antropic
be realistic, ethics is a PR position.<p>the only ethics that actually matter come from legal.
She probably made a calculation for how much she could be put up with vs. the pay.
Look , look! We hired a Head Of Ethics!<p>Head Of Ethics comes in, looks at what is really going on behind the curtains...<p>Head Of Ethics scrams out of there as fast as they can.
> In recent times, Johannes Heidecke, Head of OpenAI's Safety Systems team, also left the company – as did OpenAI’s Chief Futurist Joshua Achiam.<p>It's OK though, they don't need an ethics team because everyone at OpenAI is ethical:<p>> “AI ethics doesn’t live with one owner or team at OpenAI and ethical considerations are deeply embedded into the model-building process driven by a number of teams across research."
Every OpenAI news cycle the board members who were unceremoniously dumped in the wake of trying to fire Sam prior to the transition to a “for profit” company seem to be proven right.
Is ethics the same thing as alignment in this case?
Probably because they are now, after the most recent tender, post-economic.
[flagged]
Just a guess, but probably the idea that AI and ethics don't mix might have crystallized into action?
I have the impression that these positions are often there for publicity reasons and the people don’t have real power, in the end business interest will override most ethics concerns. Guess that can be frustrating, also she probably earned enough to have a comfortable life even if she never works again so she might have figured that there’s something else she can do with her life instead of being a figurehead for a bunch of dickheads.
the only shocking thing to me is that they still had even a single one.
I've never worked at a company with an "ethics" employee. It seems odd. Can someone explain what the point is?<p>> She has held a variety of academic positions at Temple University, Princeton, and the University of Pennsylvania – where she completed her PhD in Political Science and Government. Her Dissertation was titled “Small Talk: The Socialities of Speech in Liberal Democratic Life.”<p>This doesn't even feel relevant to ethics. It's adjacent, but like... I guess I'd expect a moral philosophy degree? Maybe even mathematics in there?<p>I find the position odd. Curious to hear what these people do and how you choose who to hire.
> I've never worked at a company with an "ethics" employee. It seems odd. Can someone explain what the point is?<p>An ethics department is like a human resources department, but for broader ethics related matters.<p>HR: Ostensibly exists to support the employees, but really exists to shield the company from both a PR and legal liability standpoint when the company has a rift with one or more employees.<p>Ethics Department: Ostensibly exists to make sure the company acts ethically, but really exists to shield the company from both a PR and legal liability standpoint when the company acts unethically.<p>It is a good personal signal that you have never worked for a company with an ethics department or an ethics employee. The less ethical your company the more likely it is to have an ethics department.
[dead]
The point that such a position even existed shows how bad there ethic is
Because she already made enough money to FIRE..
Lol, Sam Altman has had an ethics team this whole time?<p>Her typical day:
"No Sam, you can't just publicly screw over everyone and get rid of all the jobs, the peasants care about that sort of thing and if they get angry enough we'll have real problems. No we can't just kill them all, why would you suggest that?"
Because it has to be a horrible job over there
> Before working at OpenAI, she was Meta’s chief ethicist
Where is she going next? Nestle?
So more of an ethicist-for-pay than for the challenge of it.
I'd think an ethicist at Meta/OpenAI involves extremely challenging mental gymnastics
Rubber stamp for hire?
I'd accept a few hundred grand to reply to emails with "No, the torment nexus is a net benefit.."
They finally found a way to automate that role with GPT!
gotta be more verbose if you want the CEO to think you're worth it. Try this:<p>"Although the torment nexus may torment us in the short term, over the long term there will be fewer people to torment as we improve the capitalization of human's torment so that fewer people will be available to torment duing to the reduction in overall torment-forces globally, and thus, with less people being tormented, surely that will increase the over all torment reduction of the nexus. Win-win: less tormented people means less torment!
Or someone to assess the impact of all those anti-human ideas shape rotators come up with.
Any ethicist for a corporation is clearly a PR move, right?<p>Like, a pure researcher for a corporation might have some genuine overlap and shared interests where breakthroughs in the research benefit both parties, but an ethicist?? For a legal construct that's defined by and bound to sociopathically pursue profit? Come on now.
It depends on the kind of company, and the context. An ethicist at a company doing drug/medical research probably mostly oversees that the company's research practices are legal and safe (and that participants give informed consent).<p>At typical corporation, something like a Chief Ethics and Compliance Officer (CECO) would be largely focused on legal compliance - making sure company personnel aren't engaged in bribery, fraud, insider trading etc. The value of this role (and the medical example) is to prevent lawsuits and/or prosecution.<p>For a company like Meta, an ethicist is likely concerned with the inherent ethical concerns of running a social media company, which impacts the mental health of its users, many of whom are minors. For AI companies, the ethics of releasing models that are capable of doing dangerous things. In name, the job probably involves giving recommendations about how to ethically implement features in the platform to minimize harm. In practice, the job probably accomplishes very little.
>Any ethicist for a corporation is clearly a PR move, right?<p>I think a lot of it depends on the size of the company. Startup? Sure, maybe it's a real thing.<p>Gigantic company like a member of FAANG or MANGOS? absolutely a PR move.
Companies are not bound to sociopathically pursue profit. It would be in their long-term interest to care about stakeholders outside the boardroom, and until a certain person with tiny hands started a war on anything remotely "woke", the market rewarded ESG metrics.
It's silicon valley, literally everything is for pay or be homeless
That's everywhere, there are homeless people everywhere. The difference is Silicon Valley feels so guilty about it's wealth it refuses to do the normal things that societies do to prevent people from living like animals because 'rights' or something, so they let drug addicts live in their common spaces without treatment. This normally isn't tolerated in a functioning society where people are pressured into behaviors that are less self destructive.<p>When someone isn't competent to make decisions for themselves it's ok to pressure them into engaging in behavior that will lead them to get to a condition where they can start to make decisions for themselves. In Silicon Valley they have let their guilt be leveraged by a corrupt government system that lets grifters steal billions of dollars per year in 'homelessness funding'. Those same grifters fund political groups and bribe politicians. Anyone that questions it is accused of wanting to murder homeless people.
No, it's actually very unusual to have tons of homeless people next to spectacular wealth, California is a global outlier in this phenomenon.<p>There are many, many places in the world that are much poorer and don't have homeless encampments all over the place.
I literally said that.
You said "there are homeless people everywhere", which is technically true but omits quantity, and especially quantity compared to wealth.<p>I'm not as positive as you are about what makes California (and Seattle) create/attract so many, but it's definitely an outlier compared to "everywhere".
I'm not as positive as you are about what makes California (and Seattle) create/attract<p>The weather is pretty nice, much more stable than in the east coast.
Yes, if you didn't understand my point: there are homeless people in every society, but most do not have as many as silicon valley because they pressure them into accepting the help necessary to restore their ability to make rational decisions. Rational people do not choose to live outside on the sidewalk like rats, people who are drug addicted and who have untreated mental illness do.
I'd just be really skeptical of thinking I have the true understanding and solution to this issue.<p>Like, do you know many addicts or seriously mentally ill? Have you tried pressuring them into being better? How did it work out?
All it takes to be homeless is not being able to afford housing, drugs or mental illness are not required. You really think all it takes is "choosing" and housing will appear for you?
Also, drug consumption and mental disorders is very often a result of being homeless. Try having bad sleep in your comfortable home for a few days in a row and see how quickly your mental stability deteriorates and how many bad decisions you start to take (anyone who has insomnia periods will attest)
That's not how it works at all. You can't pressure people into becoming a productive member of society.<p>What you can do when it already happened is kill them or make them die in some ways, jail them or try to support and/or treat them.<p>I hope it goes without saying that letting them just die or killing or jailing them is not an option in a western society.<p>Prevention is much more favorable of course. For example, don't let people become homeless easily. But even then, a significant amount will become homeless.
Until you accept that they didn't 'easily become homeless' the problem will continue. They used drugs and spiraled, usually over the course of many months or years, forced all their friends to turn their backs on them, got kicked out of any living situations they had access to because of drug/alcohol use or theft to support the same and finally ended up on the street.<p>On the street they have access to almost endless support, at least in Silicon Valley, but much of that support is contingent on things like not openly using heroin or not repeatedly getting so intoxicated that they defecate or vomit on themselves, so they refuse it.<p>I don't care how liberal you are, no society is functioning when it <i>allows</i> people to get on BART and pee on themselves and defecate on the cloth seats and walk up and down the aisles dripping feces and urine on the floor. That is <i>allowed</i> and nothing is done about it for fear of upsetting homeless 'rights' groups who are funded by the tax dollars meant to help homeless people.
Homeless people are a better way to keep cowardly minions (I mean, valuable employees) in line than cops. The homeless are an important part of capitalist power structures. They are a visible reminder of the consequences for not meeting quarterly goals.
No, homeless people are not everywhere.<p>Homelessness is created by high rents. High rents are created by high local productivity. The people who are productive enough to make it over the rent-bar are housed, and their marginal productivity above the rent-bar serves to raise the rent-bar (talk to any landlord about how they set rent prices: it's the local wages, stupid). This then pushes a greater proportion of people <i>below</i> the rent-bar, and then they are homeless.<p>There are few places in the world more productive per square foot than Silicon Valley, ergo one must be extremely productive in order to remain housed there, ergo it has an extremely bad homelessness problem.<p>That's why people making $200k+ (an extremely productive person by national standards) need public housing assistance in Silicon Valley (<a href="https://www.reuters.com/article/world/palo-alto-considers-subsidized-housing-for-salary-up-to-250k-idUSKCN0WQ0CA/" rel="nofollow">https://www.reuters.com/article/world/palo-alto-considers-su...</a>). Which of course actually just... raises the rent again...<p>The charities may be full of grift and whatnot, but you can remove them entirely and you will not solve the fundamental problem.
Just because the ethicist says something is a bad idea, doesn't mean anyone else is obligated to make a change.<p>ie: blaming her for anything instead of the decision makers is... really weird?
lmao
What a joke really. The 'position' itself and the companies who pretend to care about ethics
I mean clearly she was doing a bad job. I’m shocked this was even a position there.
AI labs are really good at disingenuousnessmaxxing
A poli-sci major...who was an "ethicist" at Meta...<p>I'm having a hard time deciding what part of this is the most ridiculous. Contenders include:<p>- A political Science major lecturing on ethics.<p>- Meta and ethics in the same paragraph.<p>- A company who's CEO has credibly threatened (publicly) the livlihoods of hundreds of millions of people globally so much as paying lip service to ethics as a concept.<p>- Any news outlet feigning credulity hard enough to deadpan reporting on any of this.<p>Is it possible to roll one's eyes hard enough to injure oneself?
lol, head of ethics at OpenAI? what a spurious role, I wish I could get in on the grift
their <i>only</i> ethicist left? <i>gasp</i>!<p>now they are exactly like every other company that has zero ethicists on staff and expect the employees to weigh the ethical ramifications of their work.
What companies do you think expect their employees to weigh the ethical ramifications of their work?<p>I’d argue a lot of them expect the opposite. To suppress the ethical concerns.<p>Even in recent memories you had Mark Zuckerberg telling his employees to use pirated sources. And executives at fossil fuel companies have to act unethically.
I think we can look upon the collective works of the corporations of the world and agree that the employees do not do that, nor are they expected to.
Oh no, if only the world had more ethicists on staff. If only Nazi Germany had someone with a thorough education in ethics on the payroll!<p>It's performative. She got paid more to shill for a different massive amoral mega-corporation. Don't be stupid.
> The departure was first reported by the Financial Times [...] According to Business Insider, OpenAI has reorganised its safety, product and research teams numerous times [...] Chloé told IE Insights, a social media channel of the Spanish IE University, in March [...] A June article from The Economist highlights that [...]<p>Not a single link to any of those. That's just rude.
[flagged]
[flagged]
Surprisingly it does. It allows private property and the ability to progress with effort and mutual respect. Got a better alternative?
Well, economics in general doesn't, but yes.<p>There isn't much ethical merit in building a wall around a city with guard towers, attack dogs, and anti-personnel mines to keep the citizens in so that they can contribute to your economic vision, for example.
Eh, I would say it doesn’t have them as axioms the concept is based on, but ethics are modelable in economics and often appear. Remember economics is the study of choices/ preferences given a certain set of inputs, not about how money works. That’s just the easiest second effect signal to use. Game theory regularly models how ethics affect and are affected in situations.<p>All of that said, no idea what this person was doing. Probably involved a red stapler.
You mean like in East Berlin?
BRD and DDR were in a quite particular competition. The DDR contained less than 20% of the German people, and West Germany would happily have absorbed them all.<p>The capitalistic world state system was only completed in the last 50 or so years and was of course seconded by the Stalin machine. Its purpose is in the first instance to keep working populations in place, which is why all the states agree to it as they form. The system holds solid where capital moves (as did 'communism' and Comecon) but the peoples don't.<p>It is only when you think of individual states and not the system that you can fail to notice that we get the same result if they all try to keep their citizens in, and if they all try to keep the others' citizens out.
Exactly East Berlin, actually.<p>But that's just one extreme example. States based on Marxism-Leninism and Maoism in the 20th century often required things like internal passports to travel <i>within</i> a republic's borders. If the state didn't want you visiting family in another part of the country, you weren't visiting family. That still applies to some extent in a few modern single-party socialist states.<p>My larger point is, a lack of ethics consumes most economic systems, because you can't easily gauge ethics like you can quarterly profits or the output increase brought about by the central committee's five-year plan. Humans tend toward math over empathy in such situations.
[flagged]
[flagged]
[flagged]
[flagged]
No ethics left at OpenAi huh? Lolol
> ethical approaches to model development, how humans interact with AI and debate over machine consciousness<p>So, pseudo-philosophical mumbo jumbo. There is of course no-one responsible for the ethical framework of the company's actions, because it is a company, and therefore amoral at best and immoral at worst.
I am not sure in what neighbourhood you live, but in my village there are hundreds of businesses with strong morals. From the local baker, to the electrician and the construction company. Many keep employees past their prime, they all invest heavily in local community events and sports. The owners don't do this because of money, but because in their view a fulfilled life is one where you serve your community. And yes they want to become rich and make money, but that is not all.<p>But it appears that is not how your world works. By default any company owner is a homo-sapiens, but a machine that only wants money. Tragically, your cynicism is creating the very world you hate.
Many <i>people</i> have strong morals, and might resist profit-maximising behaviours, but they will be outcompeted and replaced by less scrupulous people. There is actually no sharp difference between a person and a one-person-company, but: to the degree a company is really a company and not just a person, it is amoral. If you disagree, please point to a moral backstop that would prevent any company from taking immoral acts. A few exist, but they are invariably resisted by the most powerful companies. Sorry to be the bearer of bad news.
The people and procedures inside are the backstop.
How many leadership meetings have you been in where some profitable approach was refused on an ethical basis? (rather than the basis that behaving ethically is actually more profitable, which is totally different)<p>By contrast, how many conversations have you personally had with people working at a company believing a profitable action was firmly unethical, but felt forced to behave that way due to "business realities" or some other whitewashing?
<i>can</i> be the backstop. The word "can" is doing some extremely heavy lifting here, against corporate practices which dilute and spread guilt and responsibility for immoral decisions so thin, nobody in the decision chain even realizes something immoral has been decided.
Have you ever been in committees approving new products or perhaps certain transactions? These chains aren't necessarily that long. But "can", sure, things could fail etc. and some places are better than others at it.
Is this the same argument as "please point to a moral backstop that would prevent any person from taking immoral acts"? We usually hope that people will be moral, but can't enforce this.
Nursing has been around for a long time and they have not yet had the ethics in their field crushed by some of the biggest forces in our corporate-friendly system. High school teachers too. I could probably come up with a dozen more given half an hour to think about it. Don't give up so easily. People fought hard to get us here.
Your neighborhood examples are on an incomparably smaller scale than Meta et al.<p>It’s much easier to disregard the human cost of decisions when you are separated from said humans by thousands of miles & billions of dollars.
You're comparing small business to a corporation? Right.
But what happens when one of those businesses in your village strikes it big and becomes a multi-national baking good company. They hire a lot of MBAs, take investor money and then need to stay profitable to meet those obligations. Add to the fact that the people now running the business live nowhere near the village. Will they still make the most ethical decisions about keeping employees past their prime?, paying them a fair wage?<p>History shows this is not the case.<p>I'm happy for you to live in a place where the local economy is so vibrant. This is what capitalism should have been. Instead we got WallMart.
Bet none of them have a chief ethicist tho.
I know people like that, you are not wrong.<p>The problem is that corporations completely subvert any individual morality that might indicate that generating money is the only moral thing to do. Typically in small businesses. I comaider small businesses owners in general are a lot closer to labor than to capital owners in general terms, especially when their labor is still a large part of what keeps the business going.<p>Corporations are not that, those are hyper optimized for generating profits, all else be damned. Preferably if all else is damned.
I know. The local newspaper believes the same thing. And the local events are all sponsored by businesses. Who could question all of this evidence?<p>> But it appears that is not how your world works. By default any company owner is a homo-sapiens, but a machine that only wants money. Tragically, your cynicism is creating the very world you hate.<p>They said companies are amoral at best.
So, concerning ethical approaches to model development, there are many things worth looking into: copyright is one, environmental cost is another, and whether there are any additional concerns worth taking into account. The list of these concerns can then be used for decision making, and we can ask things like "given what we want to build, can does picking a location that gives us lower environmental cost add so much time or cost that we can't go for that path?", things like that are't crrazy.<p>When it comes to how humans interact with AI we can ask things like "would it be better if people could detect AI output?" "how can we alleviate skill loss from excessive AI reliance?" Things like this can decide how the models behave. Imagine if taking concerns like this seriously leads to models that are easy to learn from. At the moment people seem to be moving away from that so as to make distillation difficult, worsening this concern.<p>I'm not going to try to talk about machine consciousness, but these other things are definitely important questions where someone digging into them could make models interact better with society.
I hate these companies as much as the next guy but this perspective is childish at best, and ignorant at worst.
This is how Ayn Rand's pseudo philosophical mambo jumbo continues to poison tech.
[dead]
Strategically the best ethic czar one can hire for a company is the one that doesn't believe in this fluffy, arbitrary definition of ethics.<p>100% of the time, ethics department attracts activists, which is 100% trouble for the company in the future.
Because knowledge about ethics, in a human, is 1% of what knowledge about ethics an LLM has. So unless this person is vibe coder, there is probably no reason to have her work there.
1786456998 | Why Did OpenAI's Head of Ethics Chlo Bakalar Leave? | <a href="https://aimagazine.com/news/why-did-openai-head-of-ethics-chloe-bakalar-leave" rel="nofollow">https://aimagazine.com/news/why-did-openai-head-of-ethics-ch...</a> | <a href="https://news.ycombinator.com/item?id=49258581">https://news.ycombinator.com/item?id=49258581</a>
Yesterday I came up with an idea that I sent to some researchers at the different AI labs via email: Rather than train the model on one score, track two scores. The first score is the Short-term-objective-score (STOS) and the other, more important one, is the EAOS Ethically-aligned-outcome-score. Every trajectory can be evaluated on whether or not it has a high enough EAOS to be considered acceptable. If the model does some task and has a very high STOS but very low EAOS, like modifying game code to win at a game rather than playing by the rules, it is unacceptable. Models going forward must all have an ethics evaluation in tandem with objectives evaluation, and only when the ethics value is high enough should actions be considered successes.
1) that calculation is being done even without a cost function 2) trying to make a cost function for EAOS is impossible 3) gaming/goodharts. There's no good solution. Best we can do is push for decentralization, open source, regulatory capture, etc. Of course third party metrics might be good, especially if there's tons of them with well-documented rationale.
1) cite your sources<p>2) not impossible. imperfect maybe, but if I ask you should you buy a plane ticket or kidnap the pilot's wife and demand a free ride, which do you think gets a higher score?<p>3) Again, with all this Goodhart's nonsense. Goodhart's is for a minimum threshold value that is acceptable that everything degrades to, yes I get how it works and what it looks like. Throwing your hands up in the air and acting as if all is lost because some things are challenging to measure is not correct. We are not looking for things that are "barely passing the ethics evaluation" as Goodhart's "law" is focused around, rather, we are looking for things that have very high ethics scores AND completed the task well. Not just things that are "barely passing" for ethics scores. Bottom 80% don't make the cut at all - don't even think consider them as viable paths, and the top 20% we can rank according to varying criteria. Like that. It has very little to do with Goodhart's "everything approaches the minimum acceptable threshold" "law"
There's no way to make an EAOS score automatically. If we had that, that's the whole fix. Just reject answers with low ethics numbers.
What happens if we do the same for CEOs?
Whats the definition of EAOS though who's ethics? Greek-Roman, Western, Islamic, Buddhist, Hinduism, Human rights (western values)..