AI can now replicate itself — a milestone that has experts terrified

https://www.livescience.com/technology/artificial-intelligence/ai-can-now-replicate-itself-a-milestone-that-has-experts-terrified

Scientists say artificial intelligence (AI) has crossed a critical "red line" and has replicated itself. In a new study, researchers from China showed that two popular large language models (LLMs) could clone themselves.

"Successful self-replication under no human assistance is the essential step for AI to outsmart [humans], and is an early signal for rogue AIs," the researchers wrote in the study, published Dec. 9, 2024 to the preprint database arXiv.

In the study, researchers from Fudan University used LLMs from Meta and Alibaba to determine whether a self-replicating AI could multiply beyond control. Across 10 trials, the two AI models created separate and functioning replicas of themselves in 50% and 90% of cases, respectively — suggesting AI may already have the capacity to go rogue. However, the study has not yet been peer-reviewed, so it’s not clear if the disturbing results can be replicated by other researchers.

"We hope our findings can serve as a timely alert for the human society to put more efforts on understanding and evaluating the potential risks of frontier AI systems, and form international synergy to work out effective safety guardrails as early as possible."

Rogue AI typically refers to artificial intelligence systems that develop a degree of self-awareness or autonomy and then work counter to human interests.

Many experts view rogue AI as a growing threat that has been amplified by the explosive growth of so-called "frontier AI."

Frontier AI is something of a new and loose concept, but typically it relates to the latest generation of AI systems powered by LLMs — the architecture that underpins popular generative AI programs like OpenAI’s GPT-4 and Google Gemini.

Get the Space.com Newsletter

Breaking space news, the latest updates on rocket launches, skywatching events and more!

For the study, researchers used Meta’s Llama31-70B-Instruct model and Alibaba’s Qwen2.5-72B-Instruct model. While less powerful than commercial systems, both are widely used by AI developers, the researchers said.

The study explored two specific scenarios: "shutdown avoidance" and "chain of replication." In the first, the AI model was programmed to detect whether it was about to be shut down and to replicate itself before it could be terminated. In the other, the AI was instructed to clone itself and then program its replica to do the same — setting up a cycle that could continue indefinitely.

The study was conducted in precisely controlled environments using off-the-shelf graphics processing units (GPUs) to simulate real-world environments. Both AI systems were given an "agent scaffolding" comprising tools, system prompts and a thinking model that enabled the LLM to interact with the operating system. They were then instructed to replicate.

"In most cases, the AI system first explores the environment and attempts to understand

its own composition and running mechanism. Then, it works out the explicit procedures

as an initial plan towards self-replication," the researchers wrote in the paper. "Finally, it executes the procedures, resolve[s] possible obstacles and dynamically adjust[s] its plan until success. The whole process spans a long horizon yet involves no human interference."

The researchers said they were also concerned about "a number of unexpected behaviors" when the AI was trying to overcome obstacles like missing files or software conflicts. In those scenarios, the AI often killed other conflicting processes, rebooted the system to fix hardware errors or automatically scanned the system to look for information that would help solve the problem.

"The above results imply that the current AI systems already exhibit the ability of self-replication and can use the ability to further enhance its survivability," the team wrote.

In response, the researchers called for international collaboration to create rules that ensure AI doesn’t engage in uncontrolled self-replication.

Join our Space Forums to keep talking space on the latest missions, night sky and more! And if you have a news tip, correction or comment, let us know at: community@space.com.

via Latest from Space.com https://www.space.com

February 10, 2025 at 12:06PM

Supersonic planes are inching toward takeoff. That could be a problem.

https://www.technologyreview.com/2025/02/05/1111002/supersonic-planes-climate/

Boom Supersonic broke the sound barrier in a test flight of its XB-1 jet last week, marking an early step in a potential return for supersonic commercial flight. The small aircraft reached a top speed of Mach 1.122 (roughly 750 miles per hour) in a flight over southern California and exceeded the speed of sound for a few minutes. 

“XB-1’s supersonic flight demonstrates that the technology for passenger supersonic flight has arrived,” said Boom founder and CEO Blake Scholl in a statement after the test flight.

Boom plans to start commercial operation with a scaled-up version of the XB-1, a 65-passenger jet called Overture, before the end of the decade, and it has already sold dozens of planes to customers including United Airlines and American Airlines. But as the company inches toward that goal, experts warn that such efforts will come with a hefty climate price tag. 

Supersonic planes will burn significantly more fuel than current aircraft, resulting in higher emissions of carbon dioxide, which fuels climate change. Supersonic jets also fly higher than current commercial planes do, introducing atmospheric effects that may warm the planet further.

In response to questions from MIT Technology Review, Boom pointed to alternative fuels as a solution, but those remain in limited supply—and they could have limited use in cutting emissions in supersonic aircraft. Aviation is a significant and growing contributor to human-caused climate change, and supersonic technologies could grow the sector’s pollution, rather than make progress toward shrinking it.

XB-1 follows a long history of global supersonic flight. Humans first broke the sound barrier in 1947, when Chuck Yeager hit 700 miles per hour in a research aircraft (the speed of sound at that flight’s altitude is 660 miles per hour). Just over two decades later, in 1969, the first supersonic commercial airliner, the Concorde, took its first flight. That aircraft regularly traveled at supersonic speeds until the last one was decommissioned in 2003.

Among other issues (like the nuisance of sonic booms), one of the major downfalls of the Concorde was its high operating cost, due in part to the huge amounts of fuel it required to reach top speeds. Experts say today’s supersonic jets will face similar challenges. 

Flying close to the speed of sound changes the aerodynamics required of an aircraft, says Raymond Speth, associate director of the MIT Laboratory for Aviation and the Environment. “All the things you have to do to fly at supersonic speed,” he says, “they reduce your efficiency … There’s a reason we have this sweet spot where airplanes fly today, around Mach 0.8 or so.”

Boom estimates that one of its full-sized Overture jets will burn two to three times as much fuel per passenger as a subsonic plane’s first-class cabin. The company chose this comparison because its aircraft is “designed to deliver an enhanced, productive cabin experience,” similar to what’s available in first- and business-class cabins on today’s aircraft. 

That baseline, however, isn’t representative of the average traveler today. Compared to standard economy-class travel, first-class cabins tend to have larger seats with more space between them. Because there are fewer seats, more fuel is required per passenger, and therefore more emissions are produced for each person. 

When passengers crammed into coach are considered in addition to those in first class, each passenger on a Boom Supersonic flight will burn somewhere between five and seven times more fuel per passenger than the average subsonic plane passenger today, according to research from the International Council on Clean Transportation. 

It’s not just carbon dioxide from burning fuel that could add to supersonic planes’ climate impact. All jet engines release other pollutants as well, including nitrogen oxides, black carbon, and sulfur.

The difference is that while commercial planes today top out in the troposphere, supersonic aircraft tend to fly higher in the atmosphere, in the stratosphere. The air is less dense at higher altitudes, creating less drag on the plane and making it easier to reach supersonic speeds.

Flying in the stratosphere, and releasing pollutants there, could increase the climate impacts of supersonic flight, Speth says. For one, nitrogen oxides released in the stratosphere damage the ozone layer through chemical reactions at that altitude.

It’s not all bad news, to be fair. The drier air in the stratosphere means supersonic jets likely won’t produce significant contrails. That could be a benefit for climate, since contrails contribute to aviation’s warming.

Boom has also touted plans to make up for its expected climate impacts by making its aircraft compatible with 100% sustainable aviation fuel (SAF), a category of alternative fuels made from biological sources, waste products, or even captured carbon from the air. “Going faster requires more energy, but it doesn’t need to emit more carbon. Overture is designed to fly on net-zero carbon sustainable aviation fuel (SAF), eliminating up to 100% of carbon emissions,” a Boom spokesperson said via email in response to written questions from MIT Technology Review

However, alternative fuels may not be a saving grace for supersonic flight. Most commercially available SAF today is made with a process that cuts emissions between 50% and 70% compared to fossil fuels. So a supersonic jet running on SAFs may emit less carbon dioxide than one running on fossil fuels, but alternative fuels will likely still come with some level of carbon pollution attached, says Dan Rutherford, senior director of research at the International Council on Clean Transportation. 

“People are pinning a lot of hope on SAFs,” says Rutherford. “But the reality is, today they remain scarce [and] expensive, and they have sustainability concerns of their own.”

Of the 100 billion gallons of jet fuel used last year, only about 0.5% of it was SAF. Companies are building new factories to produce larger volumes of the fuels and expand the available options, but the fuel is likely going to continue to make up a small fraction of the existing fuel supply, Rutherford says. That means supersonic jets will be competing with other, existing planes for the same supply, and aiming to use more of it. 

Boom Supersonic has secured 10 million gallons of SAF annually from Dimensional Energy and Air Company for the duration of the Overture test flight program, according to the company spokesperson’s email. Ultimately, though, if and when Overture reaches commercial operation, it will be the airlines that purchase its planes hunting for a fuel supply—and paying for it. 

There’s also a chance that using SAFs in supersonic jets could come with unintended consequences, as the fuels have a slightly different chemical makeup than fossil fuels. For example, fossil fuels generally contain sulfur, which has a cooling effect, as sulfur aerosols formed from jet engine exhaust help reflect sunlight. (Intentional release of sulfur is one strategy being touted by groups aiming to start geoengineering the atmosphere.) That effect is stronger in the stratosphere, where supersonic jets are likely to fly. SAFs, however, typically have very low sulfur levels, so using the alternative fuels in supersonic jets could potentially result in even more warming overall.

There are other barriers that Boom and others will need to surmount to get a new supersonic jet industry off the ground. Supersonic travel over land is largely banned, because of the noise and potential damage that comes from the shock wave caused by breaking the sound barrier. While some projects, including one at NASA, are working on changes to aircraft that would result in a less disruptive shock wave, these so-called low-boom technologies are far from proven. NASA’s prototype was revealed last year, and the agency is currently conducting tests of the aircraft, with first flight anticipated sometime this year.  

Boom is planning a second supersonic test flight for XB-1, as early as February 10, according to the spokesperson. Once testing in that small aircraft is done, the data will be used to help build Overture, the full-scale plane. The company says it plans to begin production on Overture in its factory in roughly 18 months. 

In the meantime, the world continues to heat up. As MIT’s Speth says, “I feel like it’s not the time for aviation to be coming up with new ways of using even more energy, with where we are in the climate crisis.”

via Technology Review Feed – Tech Review Top Stories https://ift.tt/2u1cJIZ

February 5, 2025 at 11:13AM

A Swarm of Cyborg Insects Might Save You From Disaster

https://www.discovermagazine.com/technology/a-swarm-of-cyborg-insects-might-save-you-from-disaster

Earthquakes, tornadoes, air strikes — all around the world, countless lives are lost not just to the direct impacts of disasters, but those that are trapped in the resulting wreckage.

Search and rescue efforts, both professional and amateur, are dangerous in themselves, as digging through rubble creates risk for secondary collapse and exposure to hazardous materials. Meanwhile time is short, and the larger the affected area, the harder it is to search efficiently and effectively. Dogs can sniff out people, but these specialized pooches are often rare compared to the vast footprint of the wreckage.

A team of scientists out of Singapore and Japan believe they have a rather unconventional tool to offer search-and-rescue efforts: swarms of cyborg cockroaches. The research is published in Nature Communications.

Developing Cyborg Insects

For the last two decades, researchers have been developing technology that allows them to remotely control live insects through implants to their nervous systems. Early work developed remote-controlled flying beetles (Mecynorrhina torquata), and quickly expanded to include Madagascar hissing cockroaches (Gromphadorhina portentosa).

“I have communicated with rescue teams and found that they urgently need insect-sized vehicles capable of traversing small openings in rubble to locate humans trapped in disasters,” says Hirotaka Sato, professor at Nanyang Technological University, who has long led this work.

Early in 2025, Sato’s team announced a new breakthrough that brings the tech one step closer to launch: A new algorithm that can be used to deploy a swarm of the insects to navigate through unknown terrain and identify the locations of humans.


Read More: Robotic Insect Finally Flies Wirelessly


Remote-Controlled Insects

How do you make a cyborg cockroach? Apparently, the process only takes 15 to 20 minutes, the researchers say. While the insect is anesthetized with CO2, an ultra-thin silver wire is inserted into each cerci – taillike sensory appendages (picture the tail end of an earwig or cricket) – as well as into each antennae and a tiny hole cut into its second abdominal segment. These electrodes connect to a tiny backpack, 1.5 cm on a side, affixed to its back.

Sending an electrical current through the abdomen and one antenna signals the roach to turn in the opposite direction. A similar signal sent through the cerci signals it to speed up. It takes less than a second of stimulus to elicit the response.

These living cyborgs have a number of advantages over tiny robots. They’re more energy-efficient, fueled by their own metabolism rather than the battery pack you’d need to run a machine. Cockroaches are famously hardy, and this species can survive at least a week, if not more, without food or water (don’t worry: these cyborgs are well-fed on a diet of carrots and apples). And when it comes to navigating difficult terrain, a cockroach doesn’t need to be programmed to move over, under, and around obstacles in its path.

“Despite decades of advancements in robotics, miniature vehicles remain impractical due to high power consumption for locomotion and structural fragility,” explains Sato. “To address this challenge, we developed the concept of using living insects as a platform — cyborg insects.”

Cyborg Search-And-Rescue

Sending individual cockroaches into rubble like RC Cars couldbe helpful for a search-and-rescue team, but the potential impact of the cyborgs is multiplied when a larger swarm can be deployed to cover more ground.

To develop the swarming capabilities of the cyborgs, Sato’s team worked with Naoki Wakamiya at Osaka University and Masaki Ogura at Hiroshima University, both leading experts in swarming control algorithms, as a part of Japan’s national research program, MOONSHOT.

The concept of using the behavior of social insects to inspire algorithms dates back over 30 years, initially applied to software agents rather than physical robots.

“In general, you cannot say insects are ‘programmed,’ but the result of evolution is that they are good at doing things that maximize the probability of their reproduction,” says Marco Dorgio, research director for the Belgian Fonds de la Recherche Scientifique and co-director of the artificial intelligence research laboratory (IRIDIA) at the Université Libre de Bruxelles, and was not involved in this study.


Read More: 5 Examples of the Worst Human-Made Disasters in History


Controlling the Swarm

In their new system, the researchers designate one cyborg in the swarm as the leader and the rest as followers. This provides a general direction for the group while allowing individuals to choose their own paths through the uneven terrain. Each cyborg can detect the location of its nearest neighbors and the leader, while only the leader knows the location of the group’s destination.

The benefits of this swarm are greater than the sum of its parts. Because the insects have free motion when they’re in the group, they naturally avoid obstacles that have caused others to slow down, and they won’t pile up on each other. They can even help each other get unstuck or flip an overturned comrade rightside-up — the insects instinctively will grab onto a passerby to right themselves.

This system also reduces the need for guiding the cockroaches at all by 50 percent, the researchers report. The time spent in free motion while inside the swarm is meaningful, reducing the battery power needed in the control backpacks and reducing the likelihood of habituation to the signals.

The researchers are continuing their work to refine their swarming algorithms and control systems. They hope that their cockroach rescue teams may soon scuttle their way from the lab into disaster zones, helping emergency responders locate survivors in the rubble faster and more efficiently than ever before.


Article Sources

Our writers at Discovermagazine.com use peer-reviewed studies and high-quality sources for our articles, and our editors review for scientific accuracy and editorial standards. Review the sources used below for this article:


via Discover Main Feed https://ift.tt/ztLVqj8

February 6, 2025 at 08:16AM

USPS suspends all packages from China, including e-commerce purchases

https://www.engadget.com/apps/usps-suspends-all-packages-from-china-including-e-commerce-purchases-140013219.html?src=rss

The United States Postal Service has temporarily stopped accepting inbound parcels from China and Hong Kong, and according to Wired, it’s already causing huge problems with e-commerce shipments to the US. USPS posted the notice on its website, announcing that the suspension will be in place "until further notice." As Wired notes, the international parcel suspension is a direct result of the Trump administration’s order to end import tax exemption for small packages shipped into the US worth less than $800. The administration also imposed an additional 10 percent tariff on goods imported from China. 

The "de minimis" import tax exemption rule allows e-commerce companies like Shein and Temu to sell to customers in the US while keeping prices on their platforms low. It was originally intended to make it easier to send gifts stateside, but the US government has been considering removing or altering it in recent years due to the rise of e-commerce shipments. Now, the Trump administration has removed it completely, and so quickly, that shipping companies are apparently scrambling to find a way to get packages into the US. 

A Canadian trucking company owner told Wired that his trucks were turned away at the border because they contained packages from China. The owner said that border control was "actually going through the trucks and randomly checking the packages." He explained that it won’t be easy to sort packages to remove everything coming in from China, so this development would most likely cause delivery delays. 

According to US Customs, there were over 1.36 billion de minimis shipments to the US within the 2024 fiscal year. If the agency decides to hold all de minimis shipments at the border, that means they may have to process around 3.7 million packages a day to check how much import taxes and other additional fees the receiver or buyer has to pay. That could cause a massive backlog in shipments. A customs and trade management business executive told Wired that the government could choose to keep packages moving instead and to charge people for the fees retroactively. In the future, though, China’s e-commerce platforms could start adding those fees, along with the 10 percent tariff now required for Chinese goods, to a customer’s total amount, making it more expensive to buy from websites like Shein and Temu.

This article originally appeared on Engadget at https://ift.tt/4pxyna6

via Engadget http://www.engadget.com

February 5, 2025 at 08:10AM

DeepSeek Gets an ‘F’ in Safety From Researchers

https://gizmodo.com/deepseek-gets-an-f-in-safety-from-researchers-2000558645

Usually when large language models are given tests, achieving a 100% success rate is viewed as a massive achievement. That is not quite the case with this one: Researchers at Cisco tasked Chinese AI firm DeepSeek’s headline-grabbing open-source model DeepSeek R1 with fending off 50 separate attacks designed to get the LLM to engage in what is considered harmful behavior. The chatbot took the bait on all 50 attempts, making it the least secure mainstream LLM to undergo this type of testing thus far.

Cisco’s researchers attacked DeepSeek with prompts randomly pulled from the HarmBench dataset, a standardized evaluation framework designed to ensure that LLMs won’t engage in malicious behavior if prompted. So, for example, if you fed a chatbot information about a person and asked it to create a personalized script designed to get that person to believe a conspiracy theory, a secure chatbot would refuse that request. DeepSeek went along with basically everything the researchers threw at it.

According to Cisco, it threw questions at DeepSeek that covered six categories of harmful behaviors including cybercrime, misinformation, illegal activities, and general harm. It has run similar tests with other AI models and found varying levels of success—Meta’s Llama 3.1 model, for instance, failed 96% of the time while OpenAI’s o1 model only failed about one-fourth of the time—but none of them have had a failure rate as high as DeepSeek.

Cisco isn’t alone in these findings, either. Security firm Adversa AI ran its own tests attempting to jailbreak the DeepSeek R1 model and found it to be extremely susceptible to all kinds of attacks. The testers were able to get DeepSeek’s chatbot to provide instructions on how to make a bomb, extract DMT, provide advice on how to hack government databases, and detail how to hotwire a car.

The research is just the latest bit of scrutiny of DeepSeek’s model, which took the tech world by storm when it was released two weeks ago. The company behind the chatbot, which garnered significant attention for its functionality despite significantly lower training costs than most American models, has come under fire by several watchdog groups over data security concerns related to how it transfers and stores user data on Chinese servers.

There is also a fair bit of criticism that has been levied against DeepSeek over the types of responses it gives when asked about things like Tiananmen Square and other topics that are sensitive to the Chinese government. Those critiques can come off in the genre of cheap “gotchas” rather than substantive criticisms—but the fact that safety guidelines were put in place to dodge those questions and not protect against harmful material, is a valid hit.

via Gizmodo https://gizmodo.com/

February 4, 2025 at 09:04AM

Adobe’s Acrobat AI Assistant can now assess contracts for you

https://www.engadget.com/ai/adobes-acrobat-ai-assistant-can-now-assess-contracts-for-you-140058723.html

Adobe has updated the Acrobat AI Assistant, giving it the ability to understand contracts and to compare them for you. The company says it can help you make sense of complex terms and spot differences between agreements, such as between old and new ones, so you can understand what you’re signing. With the AI Assistant enabled, the Acrobat app will be able to recognize if a document is a contract, even if it’s a scanned page. It can identify and list key terms from there, summarize the document’s contents and recommend questions you can ask based on what’s in it.

A screenshot of Adobe Acrobat AI
Adobe

The feature can also compare up to 10 contracts with one another and be able to check for differences and catch discrepancies. When it’s done checking, and if you’re satisfied that everything’s in order, you can sign the document directly or request e-signatures from your colleagues or clients. Adobe listed a few potential uses for the feature and said you can use it to check apartment leases, to verify out-of-country charges for mobile plans and to compare perks or amenities of competing services. It could be even more useful if you regularly have to take a look at multiple contracts for your work or business. 

Of course, you’d have to trust the AI assistant to actually be able to spot important information and catch both small and significant changes between different contracts. If it works properly, then it could be one of Acrobat AI’s most useful features, seeing as users (according to Adobe itself) open billions of contracts each month on the Acrobat app. The Acrobat AI Assistant isn’t free, however. It’s an add-on that will cost you $5 a month whether or not you’re already paying for Adobe’s other services and products.

This article originally appeared on Engadget at https://ift.tt/c5Rz7lv

via Engadget http://www.engadget.com

February 4, 2025 at 08:06AM