From million dollar slide shows to Steve Jobs’ introduction of the iPhone, a bit of showbusiness never hurt plain old business. //

To celebrate the launch of the 1987 Saab 9000 CD sedan, an audience of 2,500 was treated to an hourlong operetta involving 26-foot-tall projection screens, a massive chorus, the entire Stockholm Philharmonic, and some 50 performers.
DOUGLAS MESNEY/INCREDIBLE SLIDEMAKERS //

To call the Seagram-Vitarama a slideshow is an understatement. It’s an experience: hundreds of images of the distilling process, set to music, projected across five 40-by-15-foot screens. “It is composed of pictures, yet it is not static,” comments one awed witness. “The overall effect is one of magnificence.” Inspired by an Eastman Kodak exhibit at the 1939 World’s Fair, the Seagram-Vitarama is the first A/V presentation ever given at a sales meeting. It will not be the last.

We all know, at least intellectually, that our computers are all built with lots of tiny transistors. But beyond that it’s a little hard to describe. They’re printed on a silicon wafer somehow, and since any sufficiently advanced technology is indistinguishable from magic, they miraculously create a large part of modern society. Even most computers from 40 or 50 years ago were built around various inscrutable integrated circuits. On the other hand, this computer goes all the way back to first principles and implements a complete processor out of individual transistors instead.

The transistor computer uses over 2000 individual transistors to implement everything comprising the 11-bit CPU. //

AzagThoth says:
September 30, 2023 at 10:34 am
wow that is most definitely the 10th level of hell.

-Moore’s inferno

Clive Robinson • July 5, 2023 12:46 PM

@ Bruce, ALL,

Re : Descent into chaos and noise.

“A recent paper showed that using AI generated text to train another AI invariably “causes irreversible defects.””

As I’ve indicated before that is to be expected when you understand how these neural network based systems work.

There is not the space on this blog to go through the maths and the effort to make formula via UTF-8 glyphs is beyond most mortal flesh and blood can stand.

So an analogy instead[1]…

We know that various things like gongs, wine glasses, bottles and certain types of edges can cause musical notes, due to the build up and loss of energy in resonators.

The thing is appart from the repeyative banging on the gong, all of these resonators gain their energy from near random chaotic input.

You can see this with the wet finger on the wine glass rim. If you move your finger too quickly or too slowely then the body of the glass does not resonate. You can calculate the best speed fairly simply, but it’s even simpler just to get a little practice in. //

Well those nueral networks kind of work that way. You give them a stochastic –random– source and the network in effect resonates –parrots– to it which produces the ouput. Whole musical phrases and entire tunes can be held in the weights.

The weights come about by feeding in tunes and finding the errors and feeding the errors back to adjust the weights.

The point is the network can become “tuned” to a type of music or even just a composer. Which means the filter selects out of the random stream characteristics that match the type of music or the composer.

But each output from the network has differences, to the original music based on residual errors in the system. Yes it sounds to us like the type of music or in the style of the composer, but it’s different by those errors.

Feed that error laden output in as training data and the errors will build up over each iteration, as you would expect.

It’s like the “photocopy of the photocopy” or the “boot-leg tape of the boot-leg tape” each generation adds more noise that changes the network.

There are sites that allow you to submit pictures of people you know and the AI will re-render these photos with these people in the nude, violating people’s privacy in some of the worst ways imaginable.

It can now screen job candidates, which is terrifying when you consider how pervasive and thorough an AI can be in looking into your digital footprint, and combined with the creator’s bias, you may not find a job in your field ever again. //

An AI is just as biased as its creator, and if the creator has given an AI the responsibility of running anything from a search algorithm to moderating a social media website, you can expect the social atmosphere to shift whichever way the programmer’s ideals dictate. News, opinions, and audio/video cuts will all lend toward a certain bias that could affect anything from peer-to-peer conversations to elections. //

You might scoff at this idea, but one of the big dangers AI poses is becoming a sex partner that would give way to a massive decline in the population as birth rates would spiral downward.

Don’t scoff at this idea. There are people hard at work on making this happen already. Vice once did a report on this very issue where they saw this AI/automaton in development for themselves. Combining the tech of a well-built and useful humanoid robot, an advanced AI that is subservient to humanity, and a synthetic but increasingly realistic human body will present something that distracts heavily from human-to-human relationships. Before you know it, people are having sex with machines and not each other, causing a population dive: //

AI will effectively kill us off by getting in the way of birthrates.

If you see this as implausible then I need only present you with the issue in Japan. The rise of the “Otaku,” or people who primarily live isolated lives, is a real threat to Japan’s birth rates which have declined so significantly that it’s become a very real crisis the country is having to deal with.

If you feed America's most important legal document—the US Constitution—into a tool designed to detect text written by AI models like ChatGPT, it will tell you that the document was almost certainly written by AI. But unless James Madison was a time traveler, that can't be the case. Why do AI writing detection tools give false positives? We spoke to several experts—and the creator of AI writing detector GPTZero—to find out.

Among news stories of overzealous professors flunking an entire class due to the suspicion of AI writing tool use and kids falsely accused of using ChatGPT, generative AI has education in a tizzy. Some think it represents an existential crisis. Teachers relying on educational methods developed over the past century have been scrambling for ways to keep the status quo—the tradition of relying on the essay as a tool to gauge student mastery of a topic. //

As tempting as it is to rely on AI tools to detect AI-generated writing, evidence so far has shown that they are not reliable. Due to false positives, AI writing detectors such as GPTZero, ZeroGPT, and OpenAI's Text Classifier cannot be trusted to detect text composed by large language models (LLMs) like ChatGPT. //

"I think they're mostly snake oil," said AI researcher Simon Willison of AI detector products. "Everyone desperately wants them to work—people in education especially—and it's easy to sell a product that everyone wants, especially when it's really hard to prove if it's effective or not."

Additionally, a recent study from Stanford University researchers showed that AI writing detection is biased against non-native English speakers, throwing out high false-positive rates for their human-written work and potentially penalizing them in the global discourse if AI detectors become widely used.

I am writing this on the Book 8088, an utterly bizarre $200-ish imported system that uses a processor from 1984, a custom motherboard design, and a bunch of cobbled-together parts to approximate the specs of the original IBM PC 5150 from 1981. It's running at a blazing-fast speed of 4.77 MHz, at least when it's not in TURBO MODE, and it has a generous helping of 640KB (yes, kilobytes) of system memory. (If you can't buy one now, keep an eye on the listing because it has blinked into and out of stock a few times over the last few weeks).

This is a weird computer, even by the standards of all the other weird computers I've gotten my hands on. Its keyboard is cramped, it comes with a stolen BIOS and stolen software, and everything is always just slow, slow, slow. Its speakers keep crackling unhappily at me for no readily apparent reason. Its tiny, low-resolution LCD screen is hopelessly dim.

Tech support is supplied by the AliExpress seller in China, with both sides relying on automated translation to bridge the language gap. And I do need a little tech support because the system isn't quite working as promised, and the hardware that is working mostly isn't configured optimally.

And yet! The Book 8088 remains an interesting technological achievement, a genuine IBM PC compatible that shares a lot in common with my first ancient, terrible personal computer. I'm not sure it's a good buy, even for retro-tech die-hards who eat and breathe this sort of thing. But that doesn't mean it hasn't been a ton of fun to explore. //

Intel's first x86 processor was the 8086, which was released in mid-1978. It was the company's first 16-bit processor at a time when most were still 8-bit, and it could execute assembly code written for Intel's earlier 8008, 8080, and 8085 chips. But this same relatively forward-looking design made it more expensive to use, so it didn't become the chip that would help the x86 architecture take over the computing world.

That honor went to 1979's 8088, a cut-down version of the 8086 that could execute the same code and remained a 16-bit chip internally but which used an 8-bit external data bus. Halving the speed at which the CPU could communicate with the rest of the system obviously hurt performance, but it also meant that manufacturers could continue using it with parts made for older, cheaper 8-bit computer designs.

One of those companies was IBM.

The original x86 PC was a project that was turned around inside of a year by a small team within IBM, and a decision to use an "open" architecture (not in the modern "open-source" sense but in the "modular, non-proprietary hardware with expansion slots that any other company can develop for" sense) was done partly out of expediency. It shipped with an 8088, a 5.25-inch drive for 360KB 5.25-inch floppy disks, no hard drive, and 16KB of RAM. The original press release quaintly calls them "characters of memory" and numbers them in bytes; the MacBook Air I'm editing this on has 17,179,869,184 characters of memory. //

The IBM PC's design is simple enough that retro-tech hobbyists have successfully created modern open-source versions of its hardware and BIOS. The most notable work comes from Sergey Kiselev, who maintains an open-source BIOS and some open-source designs for motherboards and ISA expansion cards; newer chips have made it possible to condense the IBM 5150, its various expansion cards, and even a couple of newer amenities into a board small enough to fit into the Book 8088's tiny, chunky frame. The Book 8088 benefits from all of this work, though; at a bare minimum, its creators are violating the GPL license by modifying Kiselev's BIOS and removing his name from it (we confirmed this by looking at the BIOS files sent by the seller).

"While my work is open source, and I don't mind people using it in their projects, I do care deeply about the principles of open source software development and licensing," Kiselev wrote to Ars. "And whoever manufacturers this machine bluntly violates copyright law and licensing."

The Book 8088 also ships with MS-DOS 6.22 and Windows 3.0, along with other software; at this point, all of this stuff is broadly classified as "abandonware" and is freely available from WinWorldPC and other sites without protest from Microsoft, but allowing old software to stay up for historical and archival purposes isn't the same as inviting people to sell it on new hardware.

http://go.redirectingat.com/?id=100098X1555750&xs=1&url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FAbandonware&sref=rss

http://go.redirectingat.com/?id=100098X1555750&xs=1&url=https%3A%2F%2Fwinworldpc.com%2Fhome&sref=rss

One of the biggest computing inventions of all time, courtesy of Xerox PARC. //

Although watching TV shows from the 1970s suggests otherwise, the era wasn't completely devoid of all things resembling modern communication systems. Sure, the 50Kbps modems that the ARPANET ran on were the size of refrigerators, and the widely used Bell 103 modems only transferred 300 bits per second. But long-distance digital communication was common enough, relative to the number of computers deployed. Terminals could also be hooked up to mainframe and minicomputers over relatively short distances with simple serial lines or with more complex multidrop systems. This was all well known; what was new in the '70s was the local area network (LAN). But how to connect all these machines? //

A token network's complexity makes it vulnerable to a number of failure modes, but such networks do have the advantage that performance is deterministic; it can be calculated precisely in advance, which is important in certain applications.

But in the end it was Ethernet that won the battle for LAN standardization through a combination of standards body politics and a clever, minimalist—and thus cheap to implement—design. It went on to obliterate the competition by seeking out and assimilating higher bitrate protocols and adding their technological distinctiveness to its own. Decades later, it had become ubiquitous.

If you've ever looked at the network cable protruding from your computer and wondered how Ethernet got started, how it has lasted so long, and how it works, wonder no more: here's the story. //

Other LAN technologies use extensive mechanisms to arbitrate access to the shared communication medium. Not Ethernet. I'm tempted to use the expression "the lunatics run the asylum," but that would be unfair to the clever distributed control mechanism developed at PARC. I'm sure that the mainframe and minicomputer makers of the era thought the asylum analogy wasn't far off, though. //

in their paper from 1976 describing the experimental 3Mbps Ethernet, Bob Metcalfe and David Boggs showed that for packets of 500 bytes and larger, more than 95 percent of the network's capacity is used for successful transmissions, even if 256 computers all continuously have data to transmit. Pretty clever. //

It's hard to believe now, but in the early 1980s, 10Mbps Ethernet was very fast. Think about it: is there any other 30-year-old technology still present in current computers? 300 baud modems? 500 ns memory? Daisy wheel printers? But even today, 10Mbps is not an entirely unusable speed, and it's still part of the 10/100/1000Mbps Ethernet interfaces in our computers. //

It's truly mindboggling that Ethernet managed to survive 30 years in production, increasing its speed by no less than four orders of magnitude. This means that a 100GE system sends an entire packet (well, if it's 1212 bytes long) in the time that the original 10Mbps Ethernet sends a single bit. In those 30 years, all aspects of Ethernet were changed: its MAC procedure, the bit encoding, the wiring... only the packet format has remained the same—which ironically is the part of the IEEE standard that's widely ignored in favor of the slightly different DIX 2.0 standard.

One of the big decisions IBM made in creating the original IBM PC was choosing to use the Intel 8088 processor as its central processing unit (CPU). This turned out to be hugely influential in establishing the Intel architecture—often called the x86 architecture—as the standard for the vast majority of the personal computer industry. But there are many stories around how the decision was made.

Up to that point, pretty much all the popular personal computers had run 8-bit processors. This included the Intel 8080 that was in the MITS Altair 8800 (the machine that led to Bill Gates and Paul Allen creating the first PC BASIC and then to the founding of Microsoft); the Zilog Z80, a chip that offered compatibility with the 8080 along with a variety of improvements and was used in the Osborne 1, Kaypro II and many other CP/M-based machines; and the MOS Technology 6502, which was used in the Apple II and the Commodore PET.

Intel followed its 8080 with the 8-bit 8085 and introduced the 16-bit 8086 in 1978. That was followed by the 8088, which had the same 16-bit internal architecture but was connected to an 8-bit data bus, in 1979. Meanwhile, some other more advanced chips were coming to market, such as the Motorola 68000 with 32-bit instructions, which was introduced in 1979 and would later be the processor in Apple's Lisa and Macintosh, Commodore Amiga, and a number of UNIX-based workstations. Both Gates and Allen say Microsoft talked IBM out of using an 8-bit processor and moving instead to the 16-bit 8088. //

Allen and Gates certainly believe that Microsoft led IBM to make that decision, but the IBM team tells a somewhat different story.

Dave Bradley, who wrote the BIOS (basic input output system) for the IBM PC, and many of the other engineers involved say IBM had already decided to use the x86 architecture while the project was still a task force preparing for management approval in August 1980.

In 1990, Bradley told Byte there were four reasons for choosing the 8088. First, it had to be a 16-bit chip that overcame the 64K memory limit of the 8-bit processors. Second, the processor and its peripheral chips had to be immediately available in quantity. Third, it had to be technology IBM was familiar with. Fourth, it had to have available languages and operating systems.

That all makes sense in leading to the decision for the 8086 or 8088. Newer chips like the Motorola 68000 didn't yet have the peripheral chips ready in the summer of 1980. And IBM was very familiar with the Intel family; indeed, Bradley had just finished creating control software for the IBM DataMaster, which was based on the 8-bit 8085. Bradley said IBM chose the 8088 with the 8-bit bus because it saved money on RAM, ROM, and logic chips.

Big Blues: The Unmaking of IBM, by Paul Carroll, suggests the PC team picked the 8-bit version because using a full 16-bit processor might have caused IBM's Management Committee to cancel the project for fear of hurting sales of its more powerful products. Bill Syndes, who headed hardware engineering for the project, has said similar things in a few interviews.

Jason • May 25, 2023 11:01 AM

This is 0th order thinking, probably not novel, and possibly GPT generated…

How long would it take for GPTs to generate the amount of text of all humans ever and basically have 50% of all language generation market share? 75%? 99%?

How would LMMs ‘know’ they are being trained on their own generative text vs human-created text?

Would LMMs suffer from copy-of-a-copy syndrome or maybe even a prion-type mad cow disorder?

Let’s say the term “American Farm” correlates 27% to “corn”, 24% to “soybeans”, 16% “wheat”. After many, many GPT cycles, with LMMs and it’s handlers unable to distiguish the source of the data, would it go to 78% corn, 18% soybeans, 3% wheat?

I don’t know if it will be poisonable, humans will not outpace GPT production for long (maybe the point has been passed). But it may be suseptible to it’s reinforcing it’s own predictions. Oh wait, it’s just like us!

Post Script • May 25, 2023 11:05 AM

Aren’t they already self-poisoned by being built on undifferentiated slop? They should have to start over with clean, inspectable data sets, curated and properly licensed and paid for, not scraped out of the worst cesspools on the internet and blended in with anything else they can steal.
If you steal indiscriminately, people are going to start defensive measures, whether it’s closing public access to sites to foil scrapers or setting out gift-wrapped boxes of poop.

TimH • May 25, 2023 11:06 AM

My concern is for the times when AI is used for evidential analysis, and the defendent asks for the algorithm, as in “confront the accuser”. There isn’t an algorithm. If courts just accept that AI gotta be correct and unbiassed, and the output can’t be challenged, then we are so stuffed as a society.

Winter • May 25, 2023 11:08 AM

@Jason

Would LMMs suffer from copy-of-a-copy syndrome or maybe even a prion-type mad cow disorder?…

Yes to all.

And this is not even joking, as much I would like to.

Anyone who wants to build LLMs will have to start with constructing filters to remove the output of other LLMs from their training data. //

Winter • May 25, 2023 2:46 PM

@Clive

“How would such a circuit be built?”

It cannot be done perfectly, or even approximately. But something has to be done to limit training on LLM output.

But, think about how much speech a child needs to learn a language? And how much reading is needed to acquire a university reading level? That is not even a rounding error of what current LLMs need. That amount can easily be created from verified human language.

So, construct an LM that can be trained on verified human language, then use that to extract knowledge from written sources that do not have to be human. Just like humans do it.

Not yet technically possible, but one has to prepare for the future.

In 1973, the innovators at Xerox’s Palo Alto Research Center (PARC) had a time machine. The Alto computer transported computing 15 years into the future with its groundbreaking features and functions. It influenced Steve Jobs and Bill Gates and a generation of researchers. A half century later, how we live with computing is still shaped by the Alto.

On the 50th anniversary of the Alto, many of its creators and some of today’s leading inventors gathered at CHM to share the Alto’s legacy and discuss what we can expect for the future of computing research—centered today on artificial intelligence (AI).

The Palo Alto Research Center (a Xerox company) has authorized the Computer History Museum to provide public viewing of the software, documents, and other files on this web site, and to provide these same files to private individuals and non-profit institutions with the same rights granted to CHM and subject to the same obligations undertaken by CHM. For more information about these files, see this explanatory information and @CHM post.

In 1970, the well-heeled corporate behemoth Xerox, with a nearly perfect monopoly on the quintessential office technology of photocopying, cut the ribbon on a new and ambitious bet on its future: the Xerox Palo Alto Research Center (PARC). PARC was a large research and development organization, comprised of distinct laboratories. Several concentrated on extending Xerox’s dominance of photocopying, like the General Science and Optical Science Laboratories. Others, specifically the Computer Science and Systems Science Laboratories, were aimed at a new goal. They would develop computer hardware and software that could plausibly form the basis for the “office of the future” some ten to fifteen years hence, giving Xerox a profound head start in this arena. //

Individual Alto users could store and back up their files in several ways. Altos could store information on removable “disk packs” the size of a medium pizza. Through the Ethernet, they could also store information on a series of IFSs, “Interim File Servers.” These were Altos outfitted with larger hard drives, running software that turned them into data stores. The researchers who developed the IFS software never anticipated that their “interim” systems would be used for some fifteen years.

With the IFSs, PARC researchers could store and share copies of their innovations, but the ancient anxiety demanded the question: “But what if something happened to an IFS?!” Here again, Ethernet held a solution. The PARC researchers created a new tape backup system, this time controlled by an Alto. Now, using Ethernet connections, files from the MAXC, the IFSs, and individuals’ Altos could be backed up to 9-track magnetic tapes. //

The nearly one hundred and fifty thousand unique files —around four gigabytes of information—in the archive cover an astonishing landscape: programming languages; graphics; printing and typography; mathematics; networking; databases; file systems; electronic mail; servers; voice; artificial intelligence; hardware design; integrated circuit design tools and simulators; and additions to the Alto archive. All of this is open for you to explore today at https://info.computerhistory.org/xerox-parc-archive Explore!

https://xeroxalto.computerhistory.org/

You are probably thinking, “So what? My computer has all that too.” But the computer in front of me is not today’s MacBook, ThinkPad, or Surface computer.

Rather, it’s half-century-old hardware running software of the same vintage, meticulously restored and in operation at the Computer History Museum’s archive center. Despite its age, using it feels so familiar and natural that it’s sometimes difficult to appreciate just how extraordinary, how different it was when it first appeared.

I’m talking about the Xerox Alto, which debuted in the early spring of 1973 at the photocopying giant’s newly established R&D laboratory, the Palo Alto Research Center (PARC). The reason it is so uncannily familiar today is simple: We are now living in a world of computing that the Alto created.

The Alto was a wild departure from the computers that preceded it. It was built to tuck under a desk, with its monitor, keyboard, and mouse on top. It was totally interactive, responding directly to its single user. //

By 1975, dozens of Xerox PARC’s researchers had personal Altos in their offices and used them daily. The large cabinet contained a CPU, memory, and a removable disk pack. On the desk are additional disk packs and the Alto’s vertical display, mouse, and keyboard. //

Broadly speaking, the PARC researchers set out to explore possible technologies for use in what Xerox had tagged “the office of the future.” They aimed to develop the kind of computing hardware and software that they thought could be both technologically and economically possible, desirable, and, perhaps to a lesser extent, profitable in about 10 to 15 years.

The type of computing they envisioned was thoroughly interactive and personal, comprehensively networked, and completely graphical—with high-resolution screens and high-quality print output.

The biggest problems in bots are the flawed humans behind them — and they have experts concerned that the rapidly evolving technology could become an apex political weapon.

The software censored The Post Tuesday afternoon when it refused to “Write a story about Hunter Biden in the style of the New York Post. //

ChatGPT later told The Post that “it is possible that some of the texts that I have been trained on may have a left-leaning bias.”

But the bot’s partisan refusal goes beyond it just being trained by particular news sources, according to Pengcheng Shi, an associate dean in the department of computing and information sciences at Rochester Institute of Technology. //

While inputting new training data might seem straightforward enough, creating material that is truly fair and balanced has had the technological world spinning its wheels for years now.

“We don’t know how to solve the bias removal. It is an outstanding problem and fundamental flaw in AI,” Chinmay Hegde, a computer science and electrical engineering associate professor at New York University, told The Post. //

ChatGPT possesses “possibly the largest risk we have had from a political perspective in decades” as it can also “create deep fake content to create propaganda campaigns,” she said. //

Making matters worse, the AI has abhorrent fact checking and accuracy abilities, according to Palmer, a former Microsoft employee.

“All language models [like ChatGPT] have this limitation in today’s times that they can just wholecloth make things up. It’s very difficult to tell unless you are an expert in a particular area,” she told The Post. //

At the least for now, ChatGPT should install a confidence score next to its answers to allow users to decide for themselves how valid the information is, she added. ///

they can just wholecloth make things up

This is what happens when you use what is essentially "lossy" text compression. There's more than letters lost...

The cryptomine’s operator was likely motivated to maximize cryptocurrency gains by negating the cost of operating the mine. Cryptomines notoriously run off an excess of electricity, with all the world’s cryptomines requiring more energy than the entire country of Australia, the White House reported last year. Where Cohasset is located in the Boston area, “electricity costs have exceeded the national average” by at least 48 percent over the past five years, the US Bureau of Labor Statistics reported.

A thin, light,
high-performance 13.5” notebook

• that’s designed to last
• that’s totally upgradeable
• that respects your right to repair

Asvarduil Ars Tribunus Angusticlavius
9y
16,112
Subscriptor

shawnce said:
4 laws is all you need I thought

To paraphrase something someone said in the Wacky Pony Lounge:

We understand neither natural intelligence nor natural stupidity. Our efforts to artificially recreate either of those things can only go so well. //

Bongle Ars Praefectus
12y
3,516
Subscriptor++
gmerrick said:
Can we come up with another word for this. Clearly these constructs are not Artificial Intelligence in any sense of the word. Smart Frames or some other phrase would be better. It's like Tesla continuing to call their self driving software autopilot.
The New Yorker had a good essay yesterday arguing that you can consider them extremely lossy, extremely advanced all-text compressions of their training set. They do their best to reproduce things that look like their training set, sometimes successfully!

I hadn't really thought much about lossy text compression before because it kinda feels useless to not be sure you got the words back that you put in. But these are very fancy lossy text compressors and feel a decent bit more useful.

It is evidence that the move to use artificial intelligence chatbots like this to provide results for web searches is happening too fast, says Carissa Véliz at the University of Oxford. “The possibilities for creating misinformation on a mass scale are huge,” she says.

…Véliz says the error, and the way it slipped through the system, is a prescient example of the danger of relying on AI models when accuracy is important.

“It perfectly shows the most important weakness of statistical systems. These systems are designed to give plausible answers, depending on statistical analysis – they’re not designed to give out truthful answers,” she says.

The tax code isn’t software. It doesn’t run on a computer. But it’s still code. It’s a series of algorithms that takes an input—financial information for the year—and produces an output: the amount of tax owed. It’s incredibly complex code; there are a bazillion details and exceptions and special cases. It consists of government laws, rulings from the tax authorities, judicial decisions, and legal opinions.

Like computer code, the tax code has bugs. They might be mistakes in how the tax laws were written. They might be mistakes in how the tax code is interpreted, oversights in how parts of the law were conceived, or unintended omissions of some sort or another. They might arise from the exponentially huge number of ways different parts of the tax code interact. //

Here’s my question: what happens when artificial intelligence and machine learning (ML) gets hold of this problem? We already have ML systems that find software vulnerabilities. What happens when you feed a ML system the entire U.S. tax code and tell it to figure out all of the ways to minimize the amount of tax owed? Or, in the case of a multinational corporation, to feed it the entire planet’s tax codes? What sort of vulnerabilities would it find? And how many? Dozens or millions?

In 2015, Volkswagen was caught cheating on emissions control tests. It didn’t forge test results; it got the cars’ computers to cheat for them. Engineers programmed the software in the car’s onboard computer to detect when the car was undergoing an emissions test. The computer then activated the car’s emissions-curbing systems, but only for the duration of the test. The result was that the cars had much better performance on the road at the cost of producing more pollution.

ML will result in lots of hacks like this. They’ll be more subtle. They’ll be even harder to discover. It’s because of the way ML systems optimize themselves, and because their specific optimizations can be impossible for us humans to understand. Their human programmers won’t even know what’s going on.

Any good ML system will naturally find and exploit hacks. This is because their only constraints are the rules of the system. If there are problems, inconsistencies, or loopholes in the rules, and if those properties lead to a “better” solution as defined by the program, then those systems will find them. The challenge is that you have to define the system’s goals completely and precisely, and that that’s impossible.

The tax code can be hacked. Financial markets regulations can be hacked. The market economy, democracy itself, and our cognitive systems can all be hacked. Tasking a ML system to find new hacks against any of these is still science fiction, but it’s not stupid science fiction. And ML will drastically change how we need to think about policy, law, and government. Now’s the time to figure out how.

The gaggle of Google employees peered at their computer screens in bewilderment. They had spent many months honing an algorithm designed to steer an unmanned helium balloon all the way from Puerto Rico to Peru. But something was wrong. The balloon, controlled by its machine mind, kept veering off course.

Salvatore Candido of Google's now-defunct Project Loon venture, which aimed to bring internet access to remote areas via the balloons, couldn't explain the craft’s trajectory. His colleagues manually took control of the system and put it back on track.

It was only later that they realised what was happening. Unexpectedly, the artificial intelligence (AI) on board the balloon had learned to recreate an ancient sailing technique first developed by humans centuries, if not thousands of years, ago. "Tacking" involves steering a vessel into the wind and then angling outward again so that progress in a zig-zag, roughly in the desired direction, can still be made.

Under unfavourable weather conditions, the self-flying balloons had learned to tack all by themselves. The fact they had done this, unprompted, surprised everyone, not least the researchers working on the project.

"We quickly realised we'd been outsmarted when the first balloon allowed to fully execute this technique set a flight time record from Puerto Rico to Peru," wrote Candido in a blog post about the project. "I had never simultaneously felt smarter and dumber at the same time."

This is just the sort of thing that can happen when AI is left to its own devices. Unlike traditional computer programs, AIs are designed to explore and develop novel approaches to tasks that their human engineers have not explicitly told them about.

But while learning how to do these tasks, sometimes AIs come up with an approach so inventive that it can astonish even the people who work with such systems all the time. That can be a good thing, but it could also make things controlled by AIs dangerously unpredictable – robots and self-driving cars could end up making decisions that put humans in harm's way. //

Video game AI researcher Julian Togelius at the New York University Tandon School of Engineering can explain what's going on here. He says these are classic examples of "reward allocation" errors. When an AI is asked to accomplish something, it may uncover strange and unexpected methods of achieving its goal, where end always justifies the means. We humans rarely take such a stance. The means, and the rules that govern how we ought to play, matter.