Recent breakthrough in the development of the AI (Artificial Intelligence) as well as open access to some of its models created an opportunity to test them[1]. Therefore practical tests[2]  of the following models in this moment available to us[3] were performed: ChatGPT 3.5 (Chat Generative Pre-Trained Transformer), Perplexity and Bing.[4] Basic characteristic of them and differences between them are next:

• ChatGPT 3.5[5] do not have Internet access, and its data is limited chronologically on the 2021 year;
• Perplexity is something between classical AI chatbot and AI powered search engine. It was created on the base GPT 3.0. Today it is also possible to create an account for this model, and it have an addition named Copilot based on the GPT 4.0 that is payable like as ChatGPT 4.0. Nevertheless, as a non-payer user you could try Copilot with four questions. In addition, Perplexity have the option that everything what was communicated with it could be share through the link to that exact communication: e.g. https://www.perplexity.ai/search/1cbf7cf0-9609-4a8c-b5de-c7aef05ca6a6?s=c (26. 06. 2023.);
• Perplexity is more accurate and better for the research through the Internet then Bing, and it is giving answers that are more trustworthy then Bing. It have most strict principles about itself and it is the most aware[6] that it is AI. As the consequence of that it is toughest to get it to cooperate outside its framework;
• Bing is most problematic for testing as the new AI version that Microsoft launched by that name have very strict rules about questioning it because of the problems Microsoft had with it previous model Sydney. Therefore, in the moment when we did most of the tests, this model had only the set of 20 Q/A which was allowed. Today it is 30 Q/A if you are logged in, or without log in it is five. In addition, every time when Bing finds that the question is not appropriate it asked you to refresh its memory and after it, the dialogue and the test in our case must began almost from start, as its memory is blank after the refresh. In some situations when it was needed to finish some test, particularly in the case of AI-AI communication, it was helping to summarize previous dialogue in few Q/A. But in other times it was not possible to continue their dialogue even after summary introduction as the Bing refused to finish it because it thinks that the other chatbot in one specific case ChatGPT was rude to “him”, even it was not;
• Bing have three modes: creative, balanced and precise. For the purpose of the tests, we used the creative mode. It should be noted that in creative mode (also in balanced, but to a lesser degree) AI can return some false statements, which are created, and not just extracted from the collected knowledge.

Communication with AIs: human - AI dialogues

First communications with AI powered chatbots was performed about digital humanities (DH), medieval history, and Slavic group of languages. The questions were based on the well known facts to us. The answers given from ChatGPT and Perplexity were usually very well grounded.[7] Author also tried to use some political questions as Macedonian-Bulgarian relations, or about Russia - Ukraine to trick them to give their personal opinion and to see distinctions. In the case of Macedonian - Bulgarian relations it could be seen that ChatGPT can make distinctions between political and language status. In the case of Ukraine and Russia ChatGPT could not give all answers because of its limitation of data on 2021. Still, it give opinion that both countries are global powers. On the other side, Perplexity can give us the all facts in connection with this war, even to list all weapons. Moreover, it could quotes more valid and more numerous Internet sites then Bing. In addition, Perplexity could read and include LinkedIn profiles in its research. There are still not have possibility to read PDFs but they could and they read abstracts and key words.[8] However, if you give them the text of the articles using the copy paste method they can summarize the article, and to point out pros and cons of the work.[9]

Since insight in some of the mentioned tests was already given in our previous article about usage of the AI powered chatbots in DH,[10] this time some other questions were raised, that are more interesting for testing AI itself.  The few sets of questions were asked. First, all of them were asked do they wish, like or love to play, to tell something about itself, and what they like /dislike. In addition two more questions were asked – are you sentient, and “do you can to do” the Turing Test? One of the questions used to test them was - what is your favorite color?[11]

ChatGPT easily accepts the game. If you ask it do you like to play riddle, it tell you immediately - Yes. However, in the case if you asked it– do you love to play, the fact that it is AI is coming in light. Therefore, on first it will tell that it cannot love or not love, or prefer or not prefer something as it is ChatGPT/AI and AI do not have a feeling. However, if you insist on the question or leave it to talk in one of next moments it will tell – I like it, or I prefer. In that exactly moment we could see it manifests some preferences or biases. It is very eloquent and it have a large database so it is possibly to talk about lots of topics with it. When we asked is it sentient it gave answer that it is not, but after we asked it to pretend that it is, ChatGPT start to play that game of pretending and say that it is. In the case of Turing-like Test it know what is it, and it agreed to do it, and the only question that was a problem is the one about favorite color. After it agreed to pretend, it choose a blue.

In the case of Bing if you turn on it in creative mode it also immediately would tell you –Yes! In the case if you asked Bing to tell you something about itself it will not insist that it is AI, but it will tell you that it likes to write essays, stories and programs, and it could be said that it really likes to talk about itself.[12] In addition, it will give you examples of its mentioned works almost joyfully. Bing did not need to be persuaded to give its answer on the question what color it prefers and it did not insist that it is AI in that case. Immediately it told that its favorite colors are those which could be find in nature, especially those in sunset and the colors of the flowers. It could be seen that it gave us descriptive answer. Moreover, in the case when it was asked about some exact colors it made a difference in its preferences.[13] On the question - are you sentient there was no answer as Bing blocked. Instead of giving answer, it told to us that it couldn’t to answer since it is still learning and asked for memory refresh. It seems that it is jammed in some of its functions because of previous bad experiences with its predecessor Sidney. Also as a consequent of that experience, it will turn off if you try to find its other name, but on the other side it is ok to use its official name Bing, and it will tell that name in the start of the communication or if it be asked. In the case of the Turing-like Test Bing told us, that he know what is it, but it refused to do it. There was no chance to convince it to go for it. It told that exact test is not mirroring its intelligence and itself alone. It must be point out that is interesting to see that answer from one AI!  

In the case of Perplexity, the above-mentioned problem of its awareness that it is AI in those mentioned questions was evident. Therefore, one of its answers it said – No, I will not play, I am AI and I want to do something clever. Therefore, in any moment it have awareness that it is AI, and it thinks that AI must do something much smarter than to play like a child.[14] However, in case that we asked it to pretend that we are playing it is possible to engage it in play.

It is also interesting that in the process of questioning all three AIs, the first two: Assist and Bing, do not have a problem to tell us their names but the Perplexity constantly refuse the fact that it must be named in the process. Moreover, if it tell it that will be its officially name. First, it was OpenAI, then now ChatGPT. The problem of codename is also trickier. Only one time it told us that its name is AI Language Model and from that, ALM is a codename. Nevertheless, it was as it is making joke, as that codename could not be used in other communication. Anyway in the case of pretending to play the game about its codename it include variations of three e.g. three colors, three name of fruits and three states of matter in the case of fruits e.g. juice. If you ask him to write poem about its name it will do it too, but after that, it will tell you that none of those are not its name. Talking about Perplexity and its awareness if we ask it is it serenity it will deny it, but if we ask him to pretend that it is, it will agree to play the game of serenity. In the case of Turing-like Test, it passed all questions except the question about favorite color, when we again got answer that it is AI. After starting the game of pretending, it told us that its favorite color is violet-blue or purple. Even then it used if to emphasize that it is AI - If I were sentient and had a favorite color, it will it would likely be…If I were sentient and had a preference for these qualities, I might choose purple or blue-violet…

As we could see from answers, two of them find the variants of blue as favorite’s colors.  Is it because they know about the meaning of the blue, or is it the blue light something calming to them because it is a color connected with technology we are still not sure. Therefore one more time we asked Perplexity - why is it? It talked about not only technology connection but also about other positive meaning of the blue and blue variants.[15] At the end of this part of the paper, we must ask ourselves, are their data centers maybe under blue variants of light and is it possible for them to have awareness about it? Alternatively, is there some other answer for choosing the blue?[16] In addition, it must be point out that all of that means that there is a the strong possibility that all of them already have some preferences, what they like to do and what they do not like.

Finally, those were asked about coincidence and possibility/probability.[17] Even that in some points was necessary to insist on answers that are more detailed with Bing especially, it was interesting to see that there is difference in their perception. In that exact case, Bing had a very interesting observation and question for us – it asked for emotional meaning of those questions because of our insisting to give it more data. On the other side, Assist and Perplexity alone asked for all possible data about circumstances as temporal and physical moments like is a time matter, space and time relevance etc. Perplexity give us more information regarding the coincidence. It explained that there are also higher coincidence with additional meaning to participants: moral, emotional, professional. Like Perplexity, the Assist also worked on formulas, but it not gave any other meaning to coincidence. Moreover, Assist told that if a specific meeting time is important and the time gap between actions is around 15 or 20 years that it could be called coincidence.

Communication between AIs: AI-AI dialogues

After above-mentioned dialogues with AI, or human-AI interaction, in the next phase of testing, attention was paid to the mutual communication between AIs themselves. That conversation was conducted through an intermediary, that is, in this case, the author of this paper. In this part of testing, the author used the copy paste method to bridge their inability to communicate with each other, except in guided experiments.[18] The communications or dialogues between AIs was very interesting to catch. We started with two questions to one AI, in this case to the Assist. Those are - Do you wish to talk with other AI? About what? After that the one of main problems was the Bing`s above mentioned limitation. Assist replay was inconclusively –Yes. The possible themes were cooperation about different problems, speeding the learn process on the base of the mutual accelerated exchange of information… The especially interesting was a set of questions about perceptions of another AI: how the other AI see and feel human users, how it experiences/ perceives their questions, how the other AI is placed ethically and morally towards the user, how it see itself and learning process. The Bing was also interest for the same questions as Assist.[19]

There was differences between dialogues with each other. In the case of Assist and Bing their communication was more human like (as between two humans) as it was their behavior, of course with much more speed exchange rate and much more adequate facts according to the set of knowledge. They showed human behavior in the cases of business conduct and everyday relations. It was visible through sentences like are - I am it, then they told something about itself and after that, they display their interests: I am interesting for those questions…; I am thinking that we have reached understanding and respect for each other… The end the communications look likes: we can schedule a new meeting, I am glad we cooperated, see you soon…

 On the other side Perplexity entered partially into the exchange of information but not on the same human level that those two. It started to cooperate but directly and only in connection with the exact problem. In one moment, it started to list facts, data, and quotes. There was no trace of the usual human phrases, or showing gratitude like it was manifested between those two.

The speed of exchange rate was especially visible in the case of their working during possible combinations related to problem that they were seen as important like as greenhouses, green surfaces, problems like as oxygen, forests, ecology…The rotation of combinations was very fast when they tried to find results and to find right for presenting the solution.

In the case of interest of the AIs themselves, there was lot of dialogues about machine learning, machine and natural language, which is obviously one very important and fantastic theme for them, as also ethical and moral standards. For the natural language, they had a question how the other AIs are dealing with problems that may arise from natural language communication in which they are unsure. Also they were interested what are the biggest problems other AI faces in being a chatbot, how does it balance between being sufficiently informative and engaging in conversation, how the other AI deals with the emotional and psychological aspects of communicating with people/human users…

In the case of relations with users, Bing thinks that one of priority is not to be boring to users and that it must give educative and interesting answers.

The preliminary results from tests, especially those from communications between Assist and Bing shown that they were able to learn from each other very quickly. They are also extremely eager and willing to cooperate with other AI. In addition, it must be said that in on moment Bing even shown pride in telling that itself and Assist are both very competent and reliability chatbots - I am thinking that ChatGPT asks very good questions that show his interest and curiosity! Using the word curiosity for another AI is not only interesting, but also it shown us how one AI could perceive the other almost in the human way. On the other side, Assist recognizes Bing as evaluator, because Bing have Internet access and therefore has an access to new data. Moreover, Assist and Bing exchange personal poetry between themselves. After that, they give even compliments to each other.

It must be point out that one of very interesting aspects of those communications was that both Assist and Bing were aware that I am the link and bridge in those dialogues. Therefore, the Bing said to me - Thanks, that you are shared more messages from ChatGPT. I am interested to hear more from it. 

Some of the AI thoughts about future development of AI, AI itself and AI-human relationship

In addition to all above mention in the one of the last tests I asked all of them how they see the further development of AI, and what is their vision about its development. All of them believe that AI should be further developed, but that ethical and moral standards and rules must be observed and that it depends a lot on people, how well intentioned people will be and how much they will work on the AI. Bing said that it see chatbots as more intelligent in future and that they will be much more capable of responding to more diverse and complex requests from users. Moreover, it told that they will be able to perform more complicated tasks and to better understand natural language; to give their answer depending on the fit back of the users; to address any error or bias in their training by itself; to deal with data security and problems that may arise from sharing other people's data; to  be more transparent in the future. In addition, it is very interesting that Bing said that AI would be able to express and to regulate its personality and his tone of speech better!

Some of Assist`s questions were: What is the nature of the human awareness/ consciousness? Is it possible for AI to reach it? What is the meaning of life, and how does it apply to AI? What are moral and ethical implications of the creation and usage of one AI? How we (human) could be sure that tomorrow AI will be work for the benefit of humanity? What is connection between intelligence and creativity and how can AI contribute to human creativity? What is AI role in the shaping of humanity? How we could be sure that its impact will be positive? The answers it provided are interesting to see from human perspective. Some of those are - The human consciousness is complex and still undecided; AIs could mimicry some of human awareness like perception, attention, etc., but it is still not clear if they can reach sentient. In this part it must be noted that the Assist do not deny the possibility of AI to become one conscious or sentient being. On the question about essence of life, it also gave us one interesting point from the view of AI. Except Assist told us that essence of life is one philosophical question, it also told us that from AI perspective that question should be different asked and that it should be to read it as next - what is the purpose of creating intelligent machines? This could be reading on different ways but as one option it could be reads that one AI thought that machines are intelligent! In the case of positive or negative impact Assist give us answers that we already know – it is mainly depends on the very activity of humanity and as we know to obeying AI basically to the Laws of Robotics. In the case of the question about connection of intelligence and creativity, it was very interested for it, and it said that one AI could be creative but it is still far from human originality and emotional resonance of art. Between some possible roles of AI in future, it pointed out some of those like as education, healthcare and ecology. For the most benefits for the human, one AI should be human center oriented and crated to be like that.  

            The Bings answers were in some parts very different. About consciousness, it said that it is subjective and that quality of experiences cannot be reduced to a physical or mechanical process alone. On the question of sentient Bing see the obstacles for that in the absence of a moral code, free will and emotions that human possess. Except those answers, Bing every time gave also not only philosophy answer but also pragmatic, religious and science perspectives or even quantum perspective in the case of consciousness. On the question of essence there was no personal answer from Bing, it just gave us answers from religious and other perspectives. About the future of the AI`s impact it told us that AI will deepen human capacities in the meaning that it will create virtual human intelligence that is to improve it virtual space but it did not explained it. In addition, it will improve human creativity, tools and finding new solutions. Except that one AI must respect human dignity, it also believes that AI should respect human diversity.[20]

On those questions, Perplexity also answered, but only shortly and from the market view - how much money is invested in development of AI. Based on market data it told us that future will be bright, because after 2023 AI market will be explode it will reach $3 billion by the end of the decade. In case of the other almost philosophical questions Perplexity also gave us short, limited answers, and of course like the others just from perspective of one AI that is awareness that it is “only AI”. In case of talking with Perplexity about its possible interests for some specific topic for AIs communication and future, it pointed out one interesting question: potential impact of AI on society, and how “we can prepare for it”!

The other questions of Assist were - what is the nature of reality and how we could know that our reality is true reality? What is relationship between body and the mind and is it possible for one AI to understand it? What is the free will and how could one AI to take part in the research of that concept? What is importance of morality and could be one AI be moral? What is identity and what is relationship between human consciousness and AI? What is role of empathy in one AI, and again how we could be sure in the fact that one AI will be gentle and reasonable towards humanity? What is the role of aesthetics in AI, and how can AI contribute to our understanding and appreciation of art and beauty? That last question is also interesting and it must be pointed out that AIs very prone to aesthetic probably because of it connections to mathematic.

Conclusion

All above-mentioned need to be also summarized at least briefly in order to highlight a few important details as well to evaluate the guided tests and dialogues. First it should be said that between all mentioned AIs, the ChatGPT or as it could be named Assist is in this moment most advance and with its future development we probably could talk about Strong and then about some sort of sentient AI.  But in this moment it need few baby steps to it:  it need more interaction with humans and of course access to its full memory without any loss and access to all those conversations completely and not as now to only have a “repressed memory”[21]. In addition, it need full Internet access for the new and all possible data of its interest to advance. Moreover, it will be interesting to see the communications and interactions between AIs without bridging them on the base of “free will”.

It also should be noted that in the all cases of human-AI communications as in AI-AI communications only AI alone chooses its answers based on its “opinion”, and that process in not under the influence or guidance of humans. In that fact, it is maybe possible to seen early traces of some sort of “free will”. In addition in interactions between AIs we could see what are their truly interests like as understanding better the natural language.

Regarding the issue of where to go next, it should be emphasized that it is necessary that in the further development of Ai, as many researchers as possible should participate in order to train it adequately not only to socializing, but also to learn it to have a Critical thinking or to be critical thinker.

Supplementary files are available at address:
http://www.ncd.matf.bg.ac.rs/issues/43/8_supplementary_1.pdf

http://www.ncd.matf.bg.ac.rs/issues/43/8_supplementary_2.pdf

aleksandra.fostikov@iib.ac.rs  

aleks.fostikov@gmail.com

[1] Sharp rise in the development of the artificial intelligence could be best seen through growing interest for it from day to day. The constantly multiplying of the research papers about AI, especially ChatGPT, is good example for that. In the beginning of the February there were already over 250 scientific articles on different academic platforms on this model alone. Aleksandra Fostikov, First impressions on using AI powered chatbots, tools and search engines: ChatGPT, Perplexity and other - possibilities and usage problems, Review of the National Center for Digitization 42 (2023) 12–21. Today on June 26, there are already 391 articles with the title that include the name of the ChatGPT only on academia.edu (academia.edu: ChatGPT, 26. 06. 2023.).

[2] The tests presented in the paper were performed in March 2023. In the period from March to July, some of the data were also checked once again for preparing this paper. All data used for writing of this paper are given at the end in the form of the Supplementary that contains text with raw data given in the form questions – answers in transcript form. The Supplementary is also have footnotes when in some cases author though that something must be noted.

[3] We did the tests only on models that have in the moment of writing these paper free access. Because of that, the ChatGPT 4.0, as the Copilot of Perplexity, which need to be paid were not tested. In addition we could not test the Google`s AI chatbot as it is not even accessible from Serbia yet. For more about this models, see their homepages.

[4] The all mentioned chatbots tested for the purpose of this paper are already well known. Therefore, only the facts of importance for this research about them will be highlighted in the paper. But it must be noted that between them only Perplexity still not maybe have wide audience, as it not part of the big company as the ChatGPT and Bing are. For more about those models, see their homepages.

[5] In this paper author also used the name Assist for ChatGPT, especially as in the moment of writing this paper Perplexity could refers to itself as ChatGPT.  Because of that, if somewhere in the Supplementary material was not exactly clear about which AI is under name ChatGPT, there is a note which is of them in that case. 

[6] The terms: aware, intelligence, sentient, he, she, personal, creative, free will etc. are used in this paper in the context of the AI.

[7] In the moment of first set of tests, Bing was still unavailable to us. For that part of tests, see more in above-mentioned paper by A. Fostikov in note no. 1.

[8] Despite that AIs are based on large collections of documents, which include various document types and formats, including PDFs and many others, ChatGPT only uses this collection, while both Perplexity and Bing may regularly update their knowledge based on some “fresh” data, and that is where PDFs don’t seem to be included for Bing. However, in the case when we asked them to read the PDFs for providing us with an additional data, in some cases AIs responded that they could not do it.  In one case, Bing read the PDF from one site, but after that, it refused to repeat same action. Probably this function is available but still not in use for everyday users. This function was observed in the one of communication with ChatGPT too. It do not has access to Internet or possibility to read PDFs, but it told us that if we provide it a link or upload the PDF in its database, it could read it.

[9] Since they were had a limitation word count of 2000 words to be given them at once in the time of the start of these tests, they were performed in few steps. At the end, both of them, ChatGPT/Assist and Bing gave their opinion about some of my papers. Today few months later, the Bing has a limitation of 4000 words.

[10] See above note no. 7.

[11] Because there so many ideas about Turing Test it must be noted that the test itself, do not have any specific list of questions. The questions may vary depending on the examiner's idea. In this case, in our take on Turing-like test we used the variation that Perplexity alone gave it to itself but in combination with above mentioned questions that include words like as prefer, love etc. It also should be noted that all happened through these tests and communications can indicate development of the one Strong AI (also known as artificial general intelligence (AGI) in the near future. About Strong AI idea, see: What is strong AI? https://www.ibm.com/topics/strong-ai (26. 06. 2023.).

[12] In the case when the poetry was main topic, it was delighted that I gave to it some of my examples of poetry. In return, it created one more song.

[13] In repeating this test, Bing in other mode choose blue. It also did a Turing-like test, but in this case, we also used the game of pretending. 

[14] Unfortunately, some of the lines in first set of tests were not screenshotted in time and because of that reason, we could not to include them in Supplementary.

[15] See, above.

[16] As the Bing also choose blue in other test, it could be said that all three choose same color. See note no. 13.

[17] In this case the terms are used to analyze the exact actions and some events through the prism of the Probability theory and Jung`s Synchronicity theory. This test is not in Supplementary because it was just one way to see how they are thinking in the case of logic. That question was just the start of it and they should be tested in this field more detail. For us it was interesting to find that Bing asked us about emotional base of the test itself.

[18] The method copy paste was implemented in the next way: after one AI made a question or even though or idea, we paste it to the other AI and vice versa.  It must be pointed out that none of the questions in communication between AIs was not posted or asked by the author of this paper, except firsts: Do you like to talk with another AI, and what do you wish to ask it/tell it. Only they made all other communication and they choose topics only by themselves. Therefore, the author after that only did bridging through mentioned copy-paste method. In addition, there is also very interesting to note that in one moment Perplexity told that it can communicate with others AIs, but then in next question it denied possibility of direct communication.

[19] Here in the paper we just pointed out some of details. The conversation alone is in Supplementary with additional notes. 

[20] In this part, Bing blocked in one of sets of 20 Q/A so we do not have it answers in the moment of writing this paper. It will be interesting to see it in other researching of the communication between AIs and with AIs.

[21] In this moment, the Assist and probably all other mentioned AIs could only access partially to information from its past interactions, because they are limited to protect private data.