Artificial General Intelligence

Artificial basic intelligence (AGI) is a kind of synthetic intelligence (AI) that matches or exceeds human cognitive abilities throughout a vast array of cognitive tasks.

Artificial basic intelligence (AGI) is a type of expert system (AI) that matches or goes beyond human cognitive capabilities across a broad range of cognitive jobs. This contrasts with narrow AI, which is limited to specific jobs. [1] Artificial superintelligence (ASI), on the other hand, refers to AGI that significantly goes beyond human cognitive capabilities. AGI is considered one of the definitions of strong AI.


Creating AGI is a main objective of AI research study and of companies such as OpenAI [2] and Meta. [3] A 2020 study identified 72 active AGI research study and advancement jobs across 37 countries. [4]

The timeline for accomplishing AGI remains a subject of continuous debate amongst scientists and specialists. As of 2023, some argue that it may be possible in years or decades; others keep it might take a century or longer; a minority think it might never be achieved; and another minority declares that it is already here. [5] [6] Notable AI researcher Geoffrey Hinton has expressed concerns about the fast development towards AGI, suggesting it might be attained faster than lots of expect. [7]

There is dispute on the precise definition of AGI and concerning whether modern-day big language designs (LLMs) such as GPT-4 are early types of AGI. [8] AGI is a common topic in sci-fi and futures studies. [9] [10]

Contention exists over whether AGI represents an existential risk. [11] [12] [13] Many experts on AI have actually stated that reducing the threat of human extinction positioned by AGI should be an international top priority. [14] [15] Others discover the advancement of AGI to be too remote to present such a danger. [16] [17]

Terminology


AGI is likewise known as strong AI, [18] [19] complete AI, [20] human-level AI, [5] human-level intelligent AI, or basic intelligent action. [21]

Some academic sources reserve the term "strong AI" for computer system programs that experience sentience or consciousness. [a] On the other hand, weak AI (or narrow AI) is able to solve one particular problem but lacks basic cognitive abilities. [22] [19] Some academic sources utilize "weak AI" to refer more broadly to any programs that neither experience consciousness nor have a mind in the very same sense as humans. [a]

Related principles consist of synthetic superintelligence and transformative AI. An artificial superintelligence (ASI) is a theoretical kind of AGI that is far more generally intelligent than human beings, [23] while the concept of transformative AI relates to AI having a big impact on society, for forum.altaycoins.com example, comparable to the farming or industrial transformation. [24]

A structure for categorizing AGI in levels was proposed in 2023 by Google DeepMind researchers. They define 5 levels of AGI: emerging, competent, specialist, virtuoso, and superhuman. For instance, a competent AGI is specified as an AI that surpasses 50% of competent adults in a vast array of non-physical tasks, and a superhuman AGI (i.e. an artificial superintelligence) is similarly defined however with a threshold of 100%. They think about big language designs like ChatGPT or LLaMA 2 to be instances of emerging AGI. [25]

Characteristics


Various popular meanings of intelligence have been proposed. One of the leading propositions is the Turing test. However, there are other widely known definitions, and some scientists disagree with the more popular techniques. [b]

Intelligence characteristics


Researchers normally hold that intelligence is needed to do all of the following: [27]

reason, usage method, resolve puzzles, and make judgments under uncertainty
represent understanding, including common sense knowledge
plan
find out
- communicate in natural language
- if needed, incorporate these skills in conclusion of any provided objective


Many interdisciplinary techniques (e.g. cognitive science, computational intelligence, and decision making) think about additional characteristics such as imagination (the capability to form unique psychological images and ideas) [28] and autonomy. [29]

Computer-based systems that exhibit much of these abilities exist (e.g. see computational imagination, automated reasoning, decision support system, robot, evolutionary computation, smart agent). There is debate about whether modern AI systems have them to a sufficient degree.


Physical traits


Other capabilities are thought about preferable in smart systems, as they might impact intelligence or aid in its expression. These include: [30]

- the capability to sense (e.g. see, hear, etc), and
- the capability to act (e.g. move and control items, change place to check out, and so on).


This includes the ability to discover and respond to danger. [31]

Although the ability to sense (e.g. see, hear, etc) and the capability to act (e.g. move and control objects, modification location to check out, vokipedia.de and so on) can be preferable for some intelligent systems, [30] these physical abilities are not strictly required for an entity to qualify as AGI-particularly under the thesis that large language models (LLMs) might currently be or end up being AGI. Even from a less optimistic point of view on LLMs, there is no firm requirement for an AGI to have a human-like type; being a silicon-based computational system is enough, provided it can process input (language) from the external world in location of human senses. This analysis lines up with the understanding that AGI has never been proscribed a specific physical embodiment and thus does not demand a capacity for locomotion or conventional "eyes and ears". [32]

Tests for human-level AGI


Several tests suggested to validate human-level AGI have been thought about, including: [33] [34]

The idea of the test is that the maker has to attempt and pretend to be a male, by answering questions put to it, and it will just pass if the pretence is reasonably persuading. A substantial part of a jury, who must not be skilled about devices, need to be taken in by the pretence. [37]

AI-complete problems


An issue is informally called "AI-complete" or "AI-hard" if it is thought that in order to resolve it, one would need to execute AGI, due to the fact that the option is beyond the capabilities of a purpose-specific algorithm. [47]

There are numerous problems that have been conjectured to need general intelligence to solve along with humans. Examples consist of computer system vision, natural language understanding, and dealing with unforeseen situations while fixing any real-world problem. [48] Even a specific task like translation needs a maker to check out and compose in both languages, follow the author's argument (reason), comprehend the context (knowledge), and faithfully recreate the author's initial intent (social intelligence). All of these problems require to be resolved all at once in order to reach human-level machine efficiency.


However, a number of these tasks can now be performed by modern big language designs. According to Stanford University's 2024 AI index, AI has reached human-level performance on lots of criteria for checking out comprehension and visual reasoning. [49]

History


Classical AI


Modern AI research began in the mid-1950s. [50] The very first generation of AI scientists were persuaded that artificial general intelligence was possible and that it would exist in just a few decades. [51] AI pioneer Herbert A. Simon composed in 1965: "machines will be capable, within twenty years, of doing any work a man can do." [52]

Their predictions were the inspiration for Stanley Kubrick and Arthur C. Clarke's character HAL 9000, who embodied what AI researchers believed they might develop by the year 2001. AI leader Marvin Minsky was a specialist [53] on the project of making HAL 9000 as sensible as possible according to the agreement predictions of the time. He said in 1967, "Within a generation ... the problem of developing 'synthetic intelligence' will considerably be resolved". [54]

Several classical AI tasks, such as Doug Lenat's Cyc project (that started in 1984), and Allen Newell's Soar job, were directed at AGI.


However, in the early 1970s, it became apparent that researchers had grossly ignored the difficulty of the project. Funding companies became hesitant of AGI and put scientists under increasing pressure to produce beneficial "used AI". [c] In the early 1980s, Japan's Fifth Generation Computer Project revived interest in AGI, setting out a ten-year timeline that included AGI objectives like "continue a casual conversation". [58] In action to this and the success of professional systems, both industry and government pumped cash into the field. [56] [59] However, confidence in AI stunningly collapsed in the late 1980s, and the goals of the Fifth Generation Computer Project were never satisfied. [60] For the second time in twenty years, AI scientists who forecasted the impending accomplishment of AGI had actually been mistaken. By the 1990s, AI scientists had a reputation for making vain promises. They ended up being hesitant to make forecasts at all [d] and avoided reference of "human level" expert system for worry of being identified "wild-eyed dreamer [s]. [62]

Narrow AI research


In the 1990s and early 21st century, mainstream AI attained commercial success and scholastic respectability by focusing on specific sub-problems where AI can produce proven results and commercial applications, such as speech recognition and recommendation algorithms. [63] These "applied AI" systems are now used thoroughly throughout the technology industry, and research in this vein is heavily moneyed in both academia and market. As of 2018 [update], development in this field was thought about an emerging trend, and a mature stage was expected to be reached in more than 10 years. [64]

At the millenium, numerous mainstream AI scientists [65] hoped that strong AI could be developed by integrating programs that fix different sub-problems. Hans Moravec composed in 1988:


I am positive that this bottom-up path to expert system will one day satisfy the conventional top-down path majority method, prepared to offer the real-world competence and the commonsense knowledge that has been so frustratingly evasive in thinking programs. Fully intelligent makers will result when the metaphorical golden spike is driven unifying the two efforts. [65]

However, even at the time, this was challenged. For instance, Stevan Harnad of Princeton University concluded his 1990 paper on the sign grounding hypothesis by stating:


The expectation has often been voiced that "top-down" (symbolic) approaches to modeling cognition will in some way fulfill "bottom-up" (sensory) approaches somewhere in between. If the grounding factors to consider in this paper stand, then this expectation is hopelessly modular and there is truly just one feasible route from sense to symbols: from the ground up. A free-floating symbolic level like the software level of a computer system will never ever be reached by this path (or vice versa) - nor is it clear why we need to even try to reach such a level, considering that it looks as if getting there would just total up to uprooting our symbols from their intrinsic meanings (therefore merely minimizing ourselves to the practical equivalent of a programmable computer system). [66]

Modern synthetic basic intelligence research study


The term "artificial general intelligence" was utilized as early as 1997, by Mark Gubrud [67] in a discussion of the ramifications of fully automated military production and operations. A mathematical formalism of AGI was proposed by Marcus Hutter in 2000. Named AIXI, the proposed AGI agent increases "the capability to satisfy objectives in a broad range of environments". [68] This type of AGI, defined by the ability to maximise a mathematical definition of intelligence instead of display human-like behaviour, [69] was likewise called universal expert system. [70]

The term AGI was re-introduced and popularized by Shane Legg and Ben Goertzel around 2002. [71] AGI research study activity in 2006 was explained by Pei Wang and Ben Goertzel [72] as "producing publications and preliminary outcomes". The very first summertime school in AGI was arranged in Xiamen, China in 2009 [73] by the Xiamen university's Artificial Brain Laboratory and OpenCog. The very first university course was given up 2010 [74] and 2011 [75] at Plovdiv University, Bulgaria by Todor Arnaudov. MIT provided a course on AGI in 2018, arranged by Lex Fridman and including a number of guest speakers.


As of 2023 [upgrade], a little number of computer researchers are active in AGI research, and many contribute to a series of AGI conferences. However, progressively more researchers are interested in open-ended learning, [76] [77] which is the idea of allowing AI to continuously find out and innovate like humans do.


Feasibility


As of 2023, the development and prospective achievement of AGI stays a subject of extreme argument within the AI neighborhood. While conventional consensus held that AGI was a distant goal, current developments have led some scientists and market figures to declare that early types of AGI might currently exist. [78] AI leader Herbert A. Simon hypothesized in 1965 that "makers will be capable, within twenty years, of doing any work a man can do". This prediction failed to come real. Microsoft co-founder Paul Allen thought that such intelligence is unlikely in the 21st century due to the fact that it would need "unforeseeable and fundamentally unforeseeable advancements" and a "clinically deep understanding of cognition". [79] Writing in The Guardian, roboticist Alan Winfield declared the gulf in between modern computing and human-level synthetic intelligence is as broad as the gulf in between current area flight and practical faster-than-light spaceflight. [80]

An additional obstacle is the absence of clearness in specifying what intelligence involves. Does it require consciousness? Must it show the ability to set objectives as well as pursue them? Is it purely a matter of scale such that if model sizes increase sufficiently, intelligence will emerge? Are facilities such as preparation, thinking, and causal understanding required? Does intelligence require explicitly replicating the brain and its specific professors? Does it require feelings? [81]

Most AI researchers believe strong AI can be achieved in the future, but some thinkers, like Hubert Dreyfus and Roger Penrose, reject the possibility of attaining strong AI. [82] [83] John McCarthy is among those who think human-level AI will be accomplished, but that today level of progress is such that a date can not precisely be forecasted. [84] AI experts' views on the expediency of AGI wax and wane. Four polls conducted in 2012 and 2013 suggested that the average quote amongst professionals for when they would be 50% confident AGI would get here was 2040 to 2050, depending upon the survey, with the mean being 2081. Of the experts, 16.5% addressed with "never ever" when asked the very same question but with a 90% self-confidence rather. [85] [86] Further current AGI development considerations can be found above Tests for verifying human-level AGI.


A report by Stuart Armstrong and Kaj Sotala of the Machine Intelligence Research Institute found that "over [a] 60-year time frame there is a strong bias towards forecasting the arrival of human-level AI as in between 15 and 25 years from the time the forecast was made". They evaluated 95 forecasts made in between 1950 and 2012 on when human-level AI will come about. [87]

In 2023, Microsoft researchers published a comprehensive examination of GPT-4. They concluded: "Given the breadth and depth of GPT-4's capabilities, our company believe that it could reasonably be seen as an early (yet still incomplete) version of a synthetic general intelligence (AGI) system." [88] Another study in 2023 reported that GPT-4 outperforms 99% of human beings on the Torrance tests of creativity. [89] [90]

Blaise Agüera y Arcas and Peter Norvig composed in 2023 that a considerable level of general intelligence has currently been accomplished with frontier designs. They composed that unwillingness to this view originates from 4 primary factors: a "healthy hesitation about metrics for AGI", an "ideological dedication to alternative AI theories or strategies", a "devotion to human (or biological) exceptionalism", or a "concern about the economic implications of AGI". [91]

2023 also marked the development of large multimodal models (large language designs capable of processing or generating multiple modalities such as text, audio, and images). [92]

In 2024, OpenAI released o1-preview, the first of a series of designs that "spend more time believing before they respond". According to Mira Murati, this capability to believe before reacting represents a brand-new, additional paradigm. It enhances design outputs by investing more computing power when creating the response, whereas the design scaling paradigm improves outputs by increasing the model size, training data and training compute power. [93] [94]

An OpenAI staff member, Vahid Kazemi, declared in 2024 that the company had achieved AGI, mentioning, "In my viewpoint, we have actually already achieved AGI and it's even more clear with O1." Kazemi clarified that while the AI is not yet "better than any human at any task", it is "much better than the majority of people at a lot of jobs." He also resolved criticisms that big language designs (LLMs) simply follow predefined patterns, comparing their knowing process to the clinical technique of observing, hypothesizing, and verifying. These declarations have actually sparked argument, as they count on a broad and non-traditional meaning of AGI-traditionally understood as AI that matches human intelligence throughout all domains. Critics argue that, while OpenAI's models demonstrate impressive versatility, they might not totally satisfy this standard. Notably, Kazemi's comments came quickly after OpenAI eliminated "AGI" from the terms of its partnership with Microsoft, triggering speculation about the company's strategic objectives. [95]

Timescales


Progress in expert system has actually traditionally gone through periods of fast progress separated by periods when development appeared to stop. [82] Ending each hiatus were essential advances in hardware, software or both to develop area for additional progress. [82] [98] [99] For instance, the hardware available in the twentieth century was not adequate to implement deep knowing, which needs big numbers of GPU-enabled CPUs. [100]

In the intro to his 2006 book, [101] Goertzel states that quotes of the time needed before a really flexible AGI is developed differ from 10 years to over a century. Since 2007 [upgrade], the agreement in the AGI research neighborhood appeared to be that the timeline discussed by Ray Kurzweil in 2005 in The Singularity is Near [102] (i.e. in between 2015 and 2045) was plausible. [103] Mainstream AI researchers have offered a large variety of viewpoints on whether development will be this fast. A 2012 meta-analysis of 95 such opinions discovered a predisposition towards forecasting that the start of AGI would happen within 16-26 years for modern-day and historical forecasts alike. That paper has actually been criticized for how it classified opinions as specialist or non-expert. [104]

In 2012, Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton established a neural network called AlexNet, which won the ImageNet competition with a top-5 test mistake rate of 15.3%, substantially much better than the second-best entry's rate of 26.3% (the conventional approach utilized a weighted sum of ratings from various pre-defined classifiers). [105] AlexNet was regarded as the preliminary ground-breaker of the existing deep learning wave. [105]

In 2017, scientists Feng Liu, Yong Shi, and Ying Liu performed intelligence tests on openly readily available and easily accessible weak AI such as Google AI, Apple's Siri, and others. At the maximum, these AIs reached an IQ worth of about 47, which corresponds around to a six-year-old kid in very first grade. A grownup concerns about 100 on average. Similar tests were performed in 2014, with the IQ rating reaching a maximum value of 27. [106] [107]

In 2020, OpenAI established GPT-3, a language model capable of performing many varied jobs without particular training. According to Gary Grossman in a VentureBeat article, while there is agreement that GPT-3 is not an example of AGI, it is considered by some to be too advanced to be classified as a narrow AI system. [108]

In the very same year, Jason Rohrer utilized his GPT-3 account to establish a chatbot, and provided a chatbot-developing platform called "Project December". OpenAI asked for modifications to the chatbot to adhere to their security standards; Rohrer detached Project December from the GPT-3 API. [109]

In 2022, DeepMind developed Gato, a "general-purpose" system efficient in performing more than 600 different jobs. [110]

In 2023, Microsoft Research published a study on an early variation of OpenAI's GPT-4, contending that it showed more basic intelligence than previous AI designs and demonstrated human-level efficiency in tasks covering several domains, such as mathematics, coding, and law. This research study sparked a debate on whether GPT-4 could be considered an early, incomplete variation of artificial general intelligence, emphasizing the need for further expedition and evaluation of such systems. [111]

In 2023, the AI scientist Geoffrey Hinton stated that: [112]

The concept that this stuff might in fact get smarter than people - a few individuals thought that, [...] But the majority of people thought it was way off. And I thought it was way off. I believed it was 30 to 50 years or even longer away. Obviously, I no longer believe that.


In May 2023, Demis Hassabis likewise said that "The development in the last few years has actually been pretty extraordinary", which he sees no factor why it would slow down, expecting AGI within a decade and even a couple of years. [113] In March 2024, Nvidia's CEO, Jensen Huang, specified his expectation that within 5 years, AI would be capable of passing any test a minimum of along with people. [114] In June 2024, the AI scientist Leopold Aschenbrenner, a previous OpenAI worker, estimated AGI by 2027 to be "noticeably possible". [115]

Whole brain emulation


While the development of transformer models like in ChatGPT is considered the most appealing course to AGI, [116] [117] whole brain emulation can work as an alternative method. With whole brain simulation, a brain model is developed by scanning and mapping a biological brain in information, and after that copying and replicating it on a computer system or another computational device. The simulation design should be adequately devoted to the original, so that it acts in virtually the exact same method as the original brain. [118] Whole brain emulation is a type of brain simulation that is discussed in computational neuroscience and neuroinformatics, and for medical research study functions. It has been discussed in artificial intelligence research study [103] as a method to strong AI. Neuroimaging innovations that could deliver the necessary in-depth understanding are enhancing rapidly, and futurist Ray Kurzweil in the book The Singularity Is Near [102] anticipates that a map of sufficient quality will end up being available on a comparable timescale to the computing power needed to replicate it.


Early approximates


For low-level brain simulation, a very powerful cluster of computers or GPUs would be needed, offered the enormous quantity of synapses within the human brain. Each of the 1011 (one hundred billion) neurons has on typical 7,000 synaptic connections (synapses) to other neurons. The brain of a three-year-old kid has about 1015 synapses (1 quadrillion). This number decreases with age, stabilizing by the adult years. Estimates differ for an adult, ranging from 1014 to 5 × 1014 synapses (100 to 500 trillion). [120] A quote of the brain's processing power, based upon a basic switch model for nerve cell activity, is around 1014 (100 trillion) synaptic updates per second (SUPS). [121]

In 1997, Kurzweil looked at numerous price quotes for the hardware required to equate to the human brain and embraced a figure of 1016 calculations per 2nd (cps). [e] (For contrast, if a "computation" was comparable to one "floating-point operation" - a procedure used to rate current supercomputers - then 1016 "calculations" would be comparable to 10 petaFLOPS, attained in 2011, while 1018 was achieved in 2022.) He utilized this figure to predict the essential hardware would be readily available at some point between 2015 and 2025, if the rapid growth in computer power at the time of writing continued.


Current research


The Human Brain Project, an EU-funded initiative active from 2013 to 2023, has actually established a particularly in-depth and openly available atlas of the human brain. [124] In 2023, researchers from Duke University carried out a high-resolution scan of a mouse brain.


Criticisms of simulation-based methods


The synthetic nerve cell model assumed by Kurzweil and used in numerous current synthetic neural network executions is simple compared with biological nerve cells. A brain simulation would likely need to capture the in-depth cellular behaviour of biological neurons, presently understood just in broad outline. The overhead presented by complete modeling of the biological, chemical, and physical information of neural behaviour (especially on a molecular scale) would need computational powers a number of orders of magnitude bigger than Kurzweil's estimate. In addition, the estimates do not represent glial cells, which are known to contribute in cognitive procedures. [125]

A fundamental criticism of the simulated brain method derives from embodied cognition theory which asserts that human embodiment is a necessary element of human intelligence and is essential to ground significance. [126] [127] If this theory is correct, any completely practical brain design will require to encompass more than simply the nerve cells (e.g., a robotic body). Goertzel [103] proposes virtual embodiment (like in metaverses like Second Life) as an option, however it is unidentified whether this would suffice.


Philosophical point of view


"Strong AI" as defined in approach


In 1980, theorist John Searle coined the term "strong AI" as part of his Chinese space argument. [128] He proposed a difference in between 2 hypotheses about expert system: [f]

Strong AI hypothesis: An expert system system can have "a mind" and "awareness".
Weak AI hypothesis: An artificial intelligence system can (only) imitate it believes and has a mind and consciousness.


The first one he called "strong" because it makes a more powerful declaration: it assumes something special has occurred to the device that exceeds those abilities that we can check. The behaviour of a "weak AI" machine would be specifically identical to a "strong AI" machine, but the latter would also have subjective mindful experience. This usage is likewise typical in scholastic AI research study and books. [129]

In contrast to Searle and mainstream AI, some futurists such as Ray Kurzweil utilize the term "strong AI" to imply "human level artificial basic intelligence". [102] This is not the exact same as Searle's strong AI, unless it is presumed that awareness is essential for human-level AGI. Academic thinkers such as Searle do not believe that holds true, and to most expert system researchers the question is out-of-scope. [130]

Mainstream AI is most thinking about how a program behaves. [131] According to Russell and Norvig, "as long as the program works, they don't care if you call it real or a simulation." [130] If the program can act as if it has a mind, then there is no need to know if it really has mind - indeed, there would be no chance to tell. For AI research, Searle's "weak AI hypothesis" is equivalent to the statement "synthetic basic intelligence is possible". Thus, according to Russell and Norvig, "most AI researchers take the weak AI hypothesis for approved, and do not care about the strong AI hypothesis." [130] Thus, for academic AI research study, "Strong AI" and "AGI" are 2 different things.


Consciousness


Consciousness can have numerous significances, and some aspects play substantial roles in sci-fi and the ethics of expert system:


Sentience (or "extraordinary awareness"): The capability to "feel" understandings or emotions subjectively, instead of the ability to reason about perceptions. Some thinkers, such as David Chalmers, use the term "consciousness" to refer specifically to phenomenal awareness, which is roughly comparable to sentience. [132] Determining why and how subjective experience occurs is called the tough issue of consciousness. [133] Thomas Nagel described in 1974 that it "feels like" something to be conscious. If we are not conscious, then it does not feel like anything. Nagel uses the example of a bat: we can sensibly ask "what does it seem like to be a bat?" However, we are not likely to ask "what does it feel like to be a toaster?" Nagel concludes that a bat seems conscious (i.e., has consciousness) however a toaster does not. [134] In 2022, a Google engineer declared that the business's AI chatbot, LaMDA, had actually attained life, though this claim was commonly challenged by other experts. [135]

Self-awareness: To have mindful awareness of oneself as a different person, especially to be consciously familiar with one's own thoughts. This is opposed to merely being the "topic of one's believed"-an os or debugger is able to be "familiar with itself" (that is, to represent itself in the very same way it represents everything else)-however this is not what people normally indicate when they use the term "self-awareness". [g]

These traits have a moral measurement. AI life would give increase to concerns of welfare and legal defense, likewise to animals. [136] Other elements of consciousness associated to cognitive capabilities are also pertinent to the concept of AI rights. [137] Figuring out how to integrate innovative AI with existing legal and social frameworks is an emergent problem. [138]

Benefits


AGI could have a wide range of applications. If oriented towards such goals, AGI could assist reduce numerous problems in the world such as appetite, poverty and health issue. [139]

AGI could enhance efficiency and performance in a lot of jobs. For example, in public health, AGI might speed up medical research study, notably versus cancer. [140] It might look after the elderly, [141] and equalize access to quick, top quality medical diagnostics. It could offer enjoyable, low-cost and tailored education. [141] The requirement to work to subsist could end up being outdated if the wealth produced is appropriately redistributed. [141] [142] This also raises the concern of the location of people in a significantly automated society.


AGI could also assist to make logical decisions, and to expect and prevent disasters. It could also help to enjoy the benefits of possibly catastrophic innovations such as nanotechnology or climate engineering, while preventing the associated dangers. [143] If an AGI's main goal is to prevent existential disasters such as human termination (which might be challenging if the Vulnerable World Hypothesis ends up being real), [144] it could take steps to dramatically lower the dangers [143] while reducing the effect of these measures on our quality of life.


Risks


Existential threats


AGI may represent numerous kinds of existential threat, which are dangers that threaten "the premature extinction of Earth-originating smart life or the permanent and drastic damage of its capacity for desirable future advancement". [145] The danger of human termination from AGI has actually been the topic of many disputes, but there is likewise the possibility that the development of AGI would lead to a permanently problematic future. Notably, it might be used to spread out and protect the set of worths of whoever develops it. If humanity still has moral blind spots similar to slavery in the past, AGI might irreversibly entrench it, preventing ethical progress. [146] Furthermore, AGI could help with mass security and indoctrination, which could be used to create a stable repressive worldwide totalitarian program. [147] [148] There is also a risk for the devices themselves. If machines that are sentient or otherwise worthwhile of moral factor to consider are mass developed in the future, participating in a civilizational course that indefinitely overlooks their well-being and interests could be an existential disaster. [149] [150] Considering how much AGI could enhance humankind's future and help in reducing other existential risks, Toby Ord calls these existential threats "an argument for visualchemy.gallery proceeding with due caution", not for "abandoning AI". [147]

Risk of loss of control and human extinction


The thesis that AI presents an existential threat for human beings, and that this risk needs more attention, is controversial however has been endorsed in 2023 by many public figures, AI researchers and CEOs of AI business such as Elon Musk, Bill Gates, Geoffrey Hinton, Yoshua Bengio, Demis Hassabis and Sam Altman. [151] [152]

In 2014, Stephen Hawking slammed extensive indifference:


So, dealing with possible futures of enormous benefits and risks, the specialists are undoubtedly doing everything possible to make sure the best result, right? Wrong. If a remarkable alien civilisation sent us a message stating, 'We'll arrive in a couple of years,' would we simply reply, 'OK, call us when you get here-we'll leave the lights on?' Probably not-but this is more or less what is occurring with AI. [153]

The possible fate of mankind has sometimes been compared to the fate of gorillas threatened by human activities. The contrast specifies that greater intelligence enabled humankind to dominate gorillas, which are now susceptible in methods that they might not have expected. As a result, the gorilla has actually become a threatened types, not out of malice, but merely as a civilian casualties from human activities. [154]

The skeptic Yann LeCun thinks about that AGIs will have no desire to control humankind which we should take care not to anthropomorphize them and interpret their intents as we would for people. He stated that individuals will not be "smart enough to design super-intelligent machines, yet unbelievably foolish to the point of providing it moronic goals with no safeguards". [155] On the other side, the idea of instrumental merging recommends that practically whatever their objectives, smart representatives will have reasons to attempt to endure and acquire more power as intermediary steps to attaining these objectives. And that this does not require having emotions. [156]

Many scholars who are worried about existential risk advocate for more research into solving the "control issue" to address the concern: what types of safeguards, algorithms, or architectures can developers carry out to increase the probability that their recursively-improving AI would continue to behave in a friendly, rather than destructive, way after it reaches superintelligence? [157] [158] Solving the control problem is complicated by the AI arms race (which might cause a race to the bottom of safety preventative measures in order to launch items before rivals), [159] and making use of AI in weapon systems. [160]

The thesis that AI can posture existential risk likewise has detractors. Skeptics typically state that AGI is not likely in the short-term, or that concerns about AGI distract from other issues related to existing AI. [161] Former Google scams czar Shuman Ghosemajumder thinks about that for many individuals beyond the innovation market, existing chatbots and LLMs are already perceived as though they were AGI, leading to additional misunderstanding and worry. [162]

Skeptics in some cases charge that the thesis is crypto-religious, with an unreasonable belief in the possibility of superintelligence replacing an illogical belief in an omnipotent God. [163] Some scientists believe that the communication projects on AI existential threat by certain AI groups (such as OpenAI, Anthropic, DeepMind, and Conjecture) might be an at effort at regulative capture and to inflate interest in their items. [164] [165]

In 2023, the CEOs of Google DeepMind, OpenAI and Anthropic, in addition to other industry leaders and scientists, released a joint declaration asserting that "Mitigating the threat of extinction from AI ought to be an international priority alongside other societal-scale dangers such as pandemics and nuclear war." [152]

Mass unemployment


Researchers from OpenAI approximated that "80% of the U.S. labor force might have at least 10% of their work tasks affected by the introduction of LLMs, while around 19% of workers might see at least 50% of their jobs affected". [166] [167] They think about workplace employees to be the most exposed, for instance mathematicians, accounting professionals or web designers. [167] AGI might have a better autonomy, ability to make decisions, to interface with other computer system tools, however likewise to control robotized bodies.


According to Stephen Hawking, the result of automation on the lifestyle will depend upon how the wealth will be rearranged: [142]

Everyone can enjoy a life of luxurious leisure if the machine-produced wealth is shared, or many people can end up badly poor if the machine-owners effectively lobby versus wealth redistribution. Up until now, the pattern appears to be toward the second option, with technology driving ever-increasing inequality


Elon Musk considers that the automation of society will need federal governments to adopt a universal fundamental income. [168]

See also


Artificial brain - Software and hardware with cognitive abilities comparable to those of the animal or human brain
AI effect
AI safety - Research area on making AI safe and helpful
AI positioning - AI conformance to the desired objective
A.I. Rising - 2018 movie directed by Lazar Bodroža
Artificial intelligence
Automated artificial intelligence - Process of automating the application of device knowing
BRAIN Initiative - Collaborative public-private research effort revealed by the Obama administration
China Brain Project
Future of Humanity Institute - Defunct Oxford interdisciplinary research centre
General game playing - Ability of expert system to play various games
Generative synthetic intelligence - AI system efficient in generating material in response to triggers
Human Brain Project - Scientific research job
Intelligence amplification - Use of information technology to enhance human intelligence (IA).
Machine principles - Moral behaviours of man-made machines.
Moravec's paradox.
Multi-task knowing - Solving several machine discovering jobs at the same time.
Neural scaling law - Statistical law in maker knowing.
Outline of artificial intelligence - Overview of and topical guide to artificial intelligence.
Transhumanism - Philosophical motion.
Synthetic intelligence - Alternate term for or type of expert system.
Transfer learning - Machine knowing method.
Loebner Prize - Annual AI competition.
Hardware for artificial intelligence - Hardware specifically created and enhanced for artificial intelligence.
Weak artificial intelligence - Form of expert system.


Notes


^ a b See listed below for the origin of the term "strong AI", and see the academic definition of "strong AI" and weak AI in the short article Chinese space.
^ AI founder John McCarthy composes: "we can not yet identify in general what sort of computational treatments we wish to call intelligent. " [26] (For a conversation of some meanings of intelligence utilized by expert system scientists, see approach of expert system.).
^ The Lighthill report specifically criticized AI's "grandiose objectives" and led the taking apart of AI research in England. [55] In the U.S., DARPA ended up being determined to fund just "mission-oriented direct research study, instead of basic undirected research". [56] [57] ^ As AI founder John McCarthy writes "it would be a terrific relief to the rest of the workers in AI if the developers of new basic formalisms would reveal their hopes in a more safeguarded form than has sometimes held true." [61] ^ In "Mind Children" [122] 1015 cps is used. More recently, in 1997, [123] Moravec argued for 108 MIPS which would roughly correspond to 1014 cps. Moravec talks in regards to MIPS, not "cps", which is a non-standard term Kurzweil presented.
^ As defined in a basic AI book: "The assertion that machines might possibly act intelligently (or, maybe better, act as if they were smart) is called the 'weak AI' hypothesis by theorists, and the assertion that machines that do so are actually thinking (instead of simulating thinking) is called the 'strong AI' hypothesis." [121] ^ Alan Turing made this point in 1950. [36] References


^ Krishna, Sri (9 February 2023). "What is artificial narrow intelligence (ANI)?". VentureBeat. Retrieved 1 March 2024. ANI is developed to perform a single job.
^ "OpenAI Charter". OpenAI. Retrieved 6 April 2023. Our mission is to ensure that artificial basic intelligence advantages all of humanity.
^ Heath, Alex (18 January 2024). "Mark Zuckerberg's new goal is developing synthetic basic intelligence". The Verge. Retrieved 13 June 2024. Our vision is to develop AI that is much better than human-level at all of the human senses.
^ Baum, Seth D. (2020 ). A Survey of Artificial General Intelligence Projects for Ethics, Risk, and Policy (PDF) (Report). Global Catastrophic Risk Institute. Retrieved 28 November 2024. 72 AGI R&D projects were recognized as being active in 2020.
^ a b c "AI timelines: What do professionals in expert system expect for the future?". Our World in Data. Retrieved 6 April 2023.
^ Metz, Cade (15 May 2023). "Some Researchers Say A.I. Is Already Here, Stirring Debate in Tech Circles". The New York City Times. Retrieved 18 May 2023.
^ "AI leader Geoffrey Hinton gives up Google and cautions of danger ahead". The New York City Times. 1 May 2023. Retrieved 2 May 2023. It is hard to see how you can prevent the bad stars from using it for bad things.
^ Bubeck, Sébastien; Chandrasekaran, Varun; Eldan, Ronen; Gehrke, Johannes; Horvitz, Eric (2023 ). "Sparks of Artificial General Intelligence: Early explores GPT-4". arXiv preprint. arXiv:2303.12712. GPT-4 shows triggers of AGI.
^ Butler, Octavia E. (1993 ). Parable of the Sower. Grand Central Publishing. ISBN 978-0-4466-7550-5. All that you touch you alter. All that you alter changes you.
^ Vinge, Vernor (1992 ). A Fire Upon the Deep. Tor Books. ISBN 978-0-8125-1528-2. The Singularity is coming.
^ Morozov, Evgeny (30 June 2023). "The True Threat of Expert System". The New York City Times. The genuine threat is not AI itself however the way we deploy it.
^ "Impressed by synthetic intelligence? Experts say AGI is following, and it has 'existential' risks". ABC News. 23 March 2023. Retrieved 6 April 2023. AGI could present existential threats to humankind.
^ Bostrom, Nick (2014 ). Superintelligence: Paths, Dangers, Strategies. Oxford University Press. ISBN 978-0-1996-7811-2. The first superintelligence will be the last creation that humankind requires to make.
^ Roose, Kevin (30 May 2023). "A.I. Poses 'Risk of Extinction,' Industry Leaders Warn". The New York City Times. Mitigating the threat of extinction from AI must be a global priority.
^ "Statement on AI Risk". Center for AI Safety. Retrieved 1 March 2024. AI specialists warn of danger of termination from AI.
^ Mitchell, Melanie (30 May 2023). "Are AI's Doomsday Scenarios Worth Taking Seriously?". The New York City Times. We are far from developing devices that can outthink us in general methods.
^ LeCun, Yann (June 2023). "AGI does not provide an existential danger". Medium. There is no factor to fear AI as an existential hazard.
^ Kurzweil 2005, p. 260.
^ a b Kurzweil, Ray (5 August 2005), "Long Live AI", Forbes, archived from the original on 14 August 2005: Kurzweil describes strong AI as "device intelligence with the full range of human intelligence.".
^ "The Age of Artificial Intelligence: George John at TEDxLondonBusinessSchool 2013". Archived from the initial on 26 February 2014. Retrieved 22 February 2014.
^ Newell & Simon 1976, This is the term they utilize for "human-level" intelligence in the physical symbol system hypothesis.
^ "The Open University on Strong and Weak AI". Archived from the original on 25 September 2009. Retrieved 8 October 2007.
^ "What is synthetic superintelligence (ASI)?|Definition from TechTarget". Enterprise AI. Retrieved 8 October 2023.
^ "Expert system is changing our world - it is on everyone to ensure that it goes well". Our World in Data. Retrieved 8 October 2023.
^ Dickson, Ben (16 November 2023). "Here is how far we are to accomplishing AGI, according to DeepMind". VentureBeat.
^ McCarthy, John (2007a). "Basic Questions". Stanford University. Archived from the original on 26 October 2007. Retrieved 6 December 2007.
^ This list of intelligent qualities is based upon the topics covered by major AI books, including: Russell & Norvig 2003, Luger & Stubblefield 2004, Poole, Mackworth & Goebel 1998 and Nilsson 1998.
^ Johnson 1987.
^ de Charms, R. (1968 ). Personal causation. New York: Academic Press.
^ a b Pfeifer, R. and Bongard J. C., How the body shapes the way we think: a new view of intelligence (The MIT Press, 2007). ISBN 0-2621-6239-3.
^ White, R. W. (1959 ). "Motivation reevaluated: The concept of skills". Psychological Review. 66 (5 ): 297-333. doi:10.1037/ h0040934. PMID 13844397. S2CID 37385966.
^ White, R. W. (1959 ). "Motivation reassessed: The concept of competence". Psychological Review. 66 (5 ): 297-333. doi:10.1037/ h0040934. PMID 13844397. S2CID 37385966.
^ Muehlhauser, Luke (11 August 2013). "What is AGI?". Machine Intelligence Research Institute. Archived from the initial on 25 April 2014. Retrieved 1 May 2014.
^ "What is Artificial General Intelligence (AGI)?|4 Tests For Ensuring Artificial General Intelligence". Talky Blog. 13 July 2019. Archived from the initial on 17 July 2019. Retrieved 17 July 2019.
^ Kirk-Giannini, Cameron Domenico; Goldstein, Simon (16 October 2023). "AI is closer than ever to passing the Turing test for 'intelligence'. What occurs when it does?". The Conversation. Retrieved 22 September 2024.
^ a b Turing 1950.
^ Turing, Alan (1952 ). B. Jack Copeland (ed.). Can Automatic Calculating Machines Be Said To Think?. Oxford: Oxford University Press. pp. 487-506. ISBN 978-0-1982-5079-1.
^ "Eugene Goostman is a real young boy - the Turing Test says so". The Guardian. 9 June 2014. ISSN 0261-3077. Retrieved 3 March 2024.
^ "Scientists challenge whether computer system 'Eugene Goostman' passed Turing test". BBC News. 9 June 2014. Retrieved 3 March 2024.
^ Jones, Cameron R.; Bergen, Benjamin K. (9 May 2024). "People can not distinguish GPT-4 from a human in a Turing test". arXiv:2405.08007 [cs.HC]
^ Varanasi, Lakshmi (21 March 2023). "AI models like ChatGPT and GPT-4 are acing everything from the bar examination to AP Biology. Here's a list of challenging tests both AI variations have passed". Business Insider. Retrieved 30 May 2023.
^ Naysmith, Caleb (7 February 2023). "6 Jobs Expert System Is Already Replacing and How Investors Can Take Advantage Of It". Retrieved 30 May 2023.
^ Turk, Victoria (28 January 2015). "The Plan to Replace the Turing Test with a 'Turing Olympics'". Vice. Retrieved 3 March 2024.
^ Gopani, Avi (25 May 2022). "Turing Test is undependable. The Winograd Schema is obsolete. Coffee is the answer". Analytics India Magazine. Retrieved 3 March 2024.
^ Bhaimiya, Sawdah (20 June 2023). "DeepMind's co-founder recommended testing an AI chatbot's capability to turn $100,000 into $1 million to determine human-like intelligence". Business Insider. Retrieved 3 March 2024.
^ Suleyman, Mustafa (14 July 2023). "Mustafa Suleyman: My new Turing test would see if AI can make $1 million". MIT Technology Review. Retrieved 3 March 2024.
^ Shapiro, Stuart C. (1992 ). "Expert System" (PDF). In Stuart C. Shapiro (ed.). Encyclopedia of Artificial Intelligence (Second ed.). New York: John Wiley. pp. 54-57. Archived (PDF) from the initial on 1 February 2016. (Section 4 is on "AI-Complete Tasks".).
^ Yampolskiy, Roman V. (2012 ). Xin-She Yang (ed.). "Turing Test as a Defining Feature of AI-Completeness" (PDF). Artificial Intelligence, Evolutionary Computation and Metaheuristics (AIECM): 3-17. Archived (PDF) from the initial on 22 May 2013.
^ "AI Index: State of AI in 13 Charts". Stanford University Human-Centered Artificial Intelligence. 15 April 2024. Retrieved 27 May 2024.
^ Crevier 1993, pp. 48-50.
^ Kaplan, Andreas (2022 ). "Expert System, Business and Civilization - Our Fate Made in Machines". Archived from the initial on 6 May 2022. Retrieved 12 March 2022.
^ Simon 1965, p. 96 priced quote in Crevier 1993, p. 109.
^ "Scientist on the Set: An Interview with Marvin Minsky". Archived from the original on 16 July 2012. Retrieved 5 April 2008.
^ Marvin Minsky to Darrach (1970 ), priced estimate in Crevier (1993, p. 109).
^ Lighthill 1973; Howe 1994.
^ a b NRC 1999, "Shift to Applied Research Increases Investment".
^ Crevier 1993, pp. 115-117; Russell & Norvig 2003, pp. 21-22.
^ Crevier 1993, p. 211, Russell & Norvig 2003, p. 24 and see likewise Feigenbaum & McCorduck 1983.
^ Crevier 1993, pp. 161-162, 197-203, 240; Russell & Norvig 2003, p. 25.
^ Crevier 1993, pp. 209-212.
^ McCarthy, John (2000 ). "Reply to Lighthill". Stanford University. Archived from the initial on 30 September 2008. Retrieved 29 September 2007.
^ Markoff, John (14 October 2005). "Behind Artificial Intelligence, a Squadron of Bright Real People". The New York Times. Archived from the original on 2 February 2023. Retrieved 18 February 2017. At its low point, some computer system scientists and software application engineers prevented the term synthetic intelligence for fear of being considered as wild-eyed dreamers.
^ Russell & Norvig 2003, pp. 25-26
^ "Trends in the Emerging Tech Hype Cycle". Gartner Reports. Archived from the initial on 22 May 2019. Retrieved 7 May 2019.
^ a b Moravec 1988, p. 20
^ Harnad, S. (1990 ). "The Symbol Grounding Problem". Physica D. 42 (1-3): 335-346. arXiv: cs/9906002. Bibcode:1990 PhyD ... 42..335 H. doi:10.1016/ 0167-2789( 90 )90087-6. S2CID 3204300.
^ Gubrud 1997
^ Hutter, Marcus (2005 ). Universal Expert System: Sequential Decisions Based Upon Algorithmic Probability. Texts in Theoretical Computer Technology an EATCS Series. Springer. doi:10.1007/ b138233. ISBN 978-3-5402-6877-2. S2CID 33352850. Archived from the original on 19 July 2022. Retrieved 19 July 2022.
^ Legg, Shane (2008 ). Machine Super Intelligence (PDF) (Thesis). University of Lugano. Archived (PDF) from the initial on 15 June 2022. Retrieved 19 July 2022.
^ Goertzel, Ben (2014 ). Artificial General Intelligence. Lecture Notes in Computer Science. Vol. 8598. Journal of Artificial General Intelligence. doi:10.1007/ 978-3-319-09274-4. ISBN 978-3-3190-9273-7. S2CID 8387410.
^ "Who coined the term "AGI"?". goertzel.org. Archived from the original on 28 December 2018. Retrieved 28 December 2018., via Life 3.0: 'The term "AGI" was promoted by ... Shane Legg, Mark Gubrud and Ben Goertzel'
^ Wang & Goertzel 2007
^ "First International Summer School in Artificial General Intelligence, Main summer season school: June 22 - July 3, 2009, OpenCog Lab: July 6-9, 2009". Archived from the original on 28 September 2020. Retrieved 11 May 2020.
^ "Избираеми дисциплини 2009/2010 - пролетен триместър" [Elective courses 2009/2010 - spring trimester] Факултет по математика и информатика [Faculty of Mathematics and Informatics] (in Bulgarian). Archived from the initial on 26 July 2020. Retrieved 11 May 2020.
^ "Избираеми дисциплини 2010/2011 - зимен триместър" [Elective courses 2010/2011 - winter trimester] Факултет по математика и информатика [Faculty of Mathematics and Informatics] (in Bulgarian). Archived from the initial on 26 July 2020. Retrieved 11 May 2020.
^ Shevlin, Henry; Vold, Karina; Crosby, Matthew; Halina, Marta (4 October 2019). "The limitations of maker intelligence: Despite development in machine intelligence, artificial general intelligence is still a significant obstacle". EMBO Reports. 20 (10 ): e49177. doi:10.15252/ embr.201949177. ISSN 1469-221X. PMC 6776890. PMID 31531926.
^ Bubeck, Sébastien; Chandrasekaran, Varun; Eldan, Ronen; Gehrke, Johannes; Horvitz, Eric; Kamar, Ece; Lee, Peter; Lee, Yin Tat; Li, Yuanzhi; Lundberg, Scott; Nori, Harsha; Palangi, Hamid; Ribeiro, Marco Tulio; Zhang, Yi (27 March 2023). "Sparks of Artificial General Intelligence: Early explores GPT-4". arXiv:2303.12712 [cs.CL]
^ "Microsoft Researchers Claim GPT-4 Is Showing "Sparks" of AGI". Futurism. 23 March 2023. Retrieved 13 December 2023.
^ Allen, Paul; Greaves, Mark (12 October 2011). "The Singularity Isn't Near". MIT Technology Review. Retrieved 17 September 2014.
^ Winfield, Alan. "Artificial intelligence will not turn into a Frankenstein's monster". The Guardian. Archived from the initial on 17 September 2014. Retrieved 17 September 2014.
^ Deane, George (2022 ). "Machines That Feel and Think: The Role of Affective Feelings and Mental Action in (Artificial) General Intelligence". Artificial Life. 28 (3 ): 289-309. doi:10.1162/ artl_a_00368. ISSN 1064-5462. PMID 35881678. S2CID 251069071.
^ a b c Clocksin 2003.
^ Fjelland, Ragnar (17 June 2020). "Why basic expert system will not be realized". Humanities and Social Sciences Communications. 7 (1 ): 1-9. doi:10.1057/ s41599-020-0494-4. hdl:11250/ 2726984. ISSN 2662-9992. S2CID 219710554.
^ McCarthy 2007b.
^ Khatchadourian, Raffi (23 November 2015). "The Doomsday Invention: Will synthetic intelligence bring us utopia or destruction?". The New Yorker. Archived from the initial on 28 January 2016. Retrieved 7 February 2016.
^ Müller, V. C., & Bostrom, N. (2016 ). Future development in expert system: A study of professional opinion. In Fundamental issues of expert system (pp. 555-572). Springer, Cham.
^ Armstrong, Stuart, and Kaj Sotala. 2012. "How We're Predicting AI-or Failing To." In Beyond AI: Artificial Dreams, modified by Jan Romportl, Pavel Ircing, Eva Žáčková, Michal Polák and Radek Schuster, 52-75. Plzeň: University of West Bohemia
^ "Microsoft Now Claims GPT-4 Shows 'Sparks' of General Intelligence". 24 March 2023.
^ Shimek, Cary (6 July 2023). "AI Outperforms Humans in Creativity Test". Neuroscience News. Retrieved 20 October 2023.
^ Guzik, Erik E.; Byrge, Christian; Gilde, Christian (1 December 2023). "The creativity of machines: AI takes the Torrance Test". Journal of Creativity. 33 (3 ): 100065. doi:10.1016/ j.yjoc.2023.100065. ISSN 2713-3745. S2CID 261087185.
^ Arcas, Blaise Agüera y (10 October 2023). "Artificial General Intelligence Is Already Here". Noema.
^ Zia, Tehseen (8 January 2024). "Unveiling of Large Multimodal Models: Shaping the Landscape of Language Models in 2024". Unite.ai. Retrieved 26 May 2024.
^ "Introducing OpenAI o1-preview". OpenAI. 12 September 2024.
^ Knight, Will. "OpenAI Announces a New AI Model, Code-Named Strawberry, That Solves Difficult Problems Step by Step". Wired. ISSN 1059-1028. Retrieved 17 September 2024.
^ "OpenAI Employee Claims AGI Has Been Achieved". Orbital Today. 13 December 2024. Retrieved 27 December 2024.
^ "AI Index: State of AI in 13 Charts". hai.stanford.edu. 15 April 2024. Retrieved 7 June 2024.
^ "Next-Gen AI: OpenAI and Meta's Leap Towards Reasoning Machines". Unite.ai. 19 April 2024. Retrieved 7 June 2024.
^ James, Alex P. (2022 ). "The Why, What, and How of Artificial General Intelligence Chip Development". IEEE Transactions on Cognitive and Developmental Systems. 14 (2 ): 333-347. arXiv:2012.06338. doi:10.1109/ TCDS.2021.3069871. ISSN 2379-8920. S2CID 228376556. Archived from th


tyronemacleod8

54 Blog posting

Komentar