A Security Lexicon for AI

And Jessica Ji, who's a senior research analyst at the Center for Security and Emerging Technology, or CSET, at Georgetown University in Washington, DC. So now let's make a start. I'm pleased to give the floor to the unitary director, Dr. Robin Geiss.

Thank you very much. Excellencies, ladies and gentlemen, good afternoon and a very warm welcome to this side event where we're launching not the lexicon, but the project that will produce the lexicon. We've been contemplating this idea of a lexicon for quite a while, and we're really happy to see that there is growing interest and also growing momentum around this project idea just recently. So many states have pointed to it. What we're really seeing is that still, even though AI is discussed just everywhere, and it was all over the place at the high-level discussions at the General Assembly, there's very different understandings as to what autonomy, security, safety and so mean. So we feel it's very timely to start developing this lexicon where we won't prescribe definitions. We will also use the opportunity of the lexicon, of course, to point to different understandings of these terms. So we're hoping to clarify the field and level the playing field, so to say, with a view to enhancing momentum and the ability to reach agreement on some of these issues around the globe. I should thank, of course, first of all, the state representatives with us here today. It's greatly appreciated you taking the time and warm thanks to our experts for being with us. For everyone else in the room, I would really encourage you to please proactively engage in the discussion when we get there. Maybe finally, just to say that the Lexicon project will be run under the umbrella, under the roof of our newly established Center of Excellence on AI, Peace and Security, which we launched this summer in Geneva. This is just one of the many activities that the center will run. Yesterday, we launched the center's forum, the only multi-stakeholder forum, really, that will work on AI, peace and security in the entire UN ecosystem, and we're already receiving expressions of interest from various parties who want to join. I'll be short and leave it here. I look forward to engaging discussions. Thank you all for coming this afternoon.

Thank you very much, Robin, for your kind words. And I'm pleased to give the floor to Ambassador Indenbosch.

Thank you for having me here. And excellencies, distinguished delegates. Let me also thank UNIDIR for bringing us together to launch work on the lexicon for artificial intelligence at a very timely moment. Anyone even remotely familiar with discussions within the UN knows that words do matter. In fact, in the GGE laws, it took us three years of hard work, and we only agreed the characterization at the very last day. So this is really not an easy task. And we have also seen in the room earlier today that we cannot even agree on the word artificial intelligence. Some of us think that it should be something different. So, yes, this is a challenge and it will not always be straightforward, but I think it's in good hands with Unity and especially within the Centre of Excellence. As said before, AI is not a new discipline, but its military applications are developing quickly, and the direction of travel in international discussion is still taking shape. UN member states approach these questions from different security perspectives and with different levels of technical expertise, and sometimes the same terms carry different meanings, even further complicating our work. Take, for instance, the term "control". Control over what? Control by whom? Control at what level? And control at which stage? Engineers, military personnel, diplomats from different member states may answer these questions differently. Something that we have also seen in the GGE laws. And those differences matter. And that's why the Netherlands strongly supports UNIDIR's initiative to be the first to develop an AI lexicon. Let me also recall here that the Netherlands is a strong supporter of UNITAR's Centre of Excellence on AI, Peace and Security. The Centre provides a place where independent research, technical knowledge and the needs of diplomacy can inform one another. By drawing from specialists from different regions and disciplines, as UNITAR has also done during the RE-AIM regional consultations, it can help Member States work through technical questions that shape policy choices. A lexicon is a clear lighthouse to make this tangible. It could clarify concepts and show where interpretation and further work is needed. For the Netherlands, this initiative also connects directly to what we have been pursuing together with the Republic of Korea and other Member States in the First Committee. Last week, together with the Republic of Korea, the Netherlands tabled for the third time a resolution on AI in the military domain. Our aim is to take the next step and move towards structured dialogues, which entail discussions with a focused agenda, sufficient depth, and an ambition to produce concrete outcomes such as confidence-building measures. And as you might have noticed, in this year's resolution, we also welcomed UNIDIR's Centre of Excellence and encouraged them and other UN entities to work on a lexicon, among other things. At last, A lexicon cannot settle political differences. Those remain for the Member States. But a lexicon can help to better understand each other and to make our discussions more productive. And I wish UNDRIP and all those contributing to this project every success. Thank you for giving me the floor.

Thank you very much, Ambassador, for your kind words and encouragement. I'm now pleased to give the floor to Ambassador Marchegiani for his remarks.

Thank you. Thank you, UNIDIR, for organizing this event. And, Excellencies, dear guests, dear friends, through its research, expertise, and capacity building, UNIDIR has become an indispensable partner, and in the field of research on AI, including. We do attach a great importance to its security and technology program. We are honored to be among the founding members of the center. And this is a good point to jump into the subject of today's event. I want to tell you a couple of words on the Italian AI strategy in the defense domain. It has been approved and released in February last year. Just a couple of features. We decided that it has to be ethical and human-centric, so grounded in international humanitarian law, with meaningful human control. And here already we meet the first subtleties. What is meaningful? What is human? What is control, as Robert was saying? The strategy aims at a strategic autonomy, so to reduce technological gaps and build domestic data power and proprietary computing capacity and secure networks. And of course, all this under a strong allied integration. Consider, for example, the word and the notion of autonomy. It's very difficult to describe it and clearly distinguish from an automated system and from an autonomous one. So it is a fact, as Robert said, we work with words. Our results or non-results are based on words. That is why Italy really welcomes the idea, the initiative to develop a security lexicon. And I don't hide, we have high expectation on this, about its readability, conciseness, and readability. Applicability, sorry. Some other features that we look with very much, with big interest as far as the lexicon by UNITAR is concerned. Neutrality. So a lexicon based on a solid scientific expertise. Inclusiveness. Indeed, a shared vocabulary would not simplify political divisions, but would simplify at least the discourse around the table, understanding each other. Adaptability, since the matter per se is strongly, is evolving by the day. And outreach. This lexicon will have to be disseminated in order to be credible. So there must be a marketing effort, a marketing campaign. The goal could be that AI, when asked about a reference lexicon, would answer the UNIDIR lexicon. I don't know whether this is doable. And in the end, all it boils down to being a reference, being authoritative. UNIDIR is the only global agency that can have this ambition. So diplomacy indeed begins with dialogue and a meaningful dialogue requires a language. I therefore wish Unity every success and count on us on this very project. Thank you.

Thank you so much, Ambassador, for your kind words and also for giving us with a few pieces of advice on how to proceed and to make this a success. And last but not least, I'm pleased to give the floor to Ambassador, please.

Thank you. Distinguished delegates, colleagues, ladies and gentlemen, good afternoon. It's my great pleasure to join this meaningful event, and I want to thank Yonidir for convening this timely discussion on the theme of AI security lexicon, a security lexicon for AI. And I want to also thank our distinguished speakers and friends, Robin, Ambassador Bush, Ambassador Marqueziani, and all of you. So UNITAR has been long been an indispensable partner in addressing emerging security challenges, especially from the AI and its new center of excellence on AI, peace and security is one, I think the best example. And it has been also been a staunch supporter of the VM process and the AIMD resolutions, which the Netherlands and our has been, you know, rallying for the past three years straight. Let me introduce what Korea has been done, and I'm repeating myself, but ever since we co-hosted the first REIM in The Hague in September 24, Korea hosted the second REIM summit. I think it has my personal honor to be able to work as an organizer for the second REIM summit in Seoul. And to include regional perspectives, we support all the convening of regional consultation process across five regions in 2025, including one in Seoul. In the United Nations General Assembly and Security Council last year in September 2025, RK was the president of the UN Security Council. And with that capacity, my president was here presiding the first, not really the first one, but very extensive UN Security Council open debate on AI and trans of peace and security. And this year we have also witnessed the French presidency in September opened another open debate on AI and its security implications, which is actually widening day by day. I'm not going to repeat all the, or request, you know, repeat the why this event is important, why the language or having the precise terminology is important, but just recall all, what you, just recall the meetings we just had in the first committee. One of the best tactics that when you don't like the other, you know, any of the speakers, if you want to really, you know, stop what any party is speaking, One of the most, and easiest, but most effective tactic is question the very concept that they're raising. What is A? What is B? Explain. It's getting impossible, and it's just one of the best, how should I put it, really sabotage tactics that you can use in the conference room. But at the same time, I just want to finish, close this very short intervention with this quote from great philosopher Immanuel Kant, and it is the critic of pure reason. It's going to be long, but please listen. Quote, "It was the duty of philosophy to destroy the illusions which had their origin in misconceptions, whatever darling hopes and valid expectations may be ruined by explanations," unquote. Hard to get, but in simple terms, we need to remove linguistic ambiguity, or otherwise, whatever debates that may follow would not add up much. As professionals in the United Nations, I think by engaging, starting this project, I think we can travel actually much further, starting from the very basics. And in that capacity, I strongly want to reiterate my support to this project and the work of the UNIDO and all the support from our friends. Thank you.

Thank you very much, Ambassador. And Indeed, as you just mentioned, a lexicon can serve important practical functions for the international community, one of which is primarily to help disambiguate terms, to enhance familiarity with AI security topics, and also to serve as a resource for states and the international community to draw on and to build common understandings. However, we fully acknowledge the development of a lexicon is a very complex endeavor. which requires addressing important methodological questions. And to help us unpack in greater depth what some of the challenges are, what the lexicon entails, what developing a lexicon entails, how to categorize AI safety vocabularies, how to build a coherent framework, we will now hear from our two technical experts today who will help us unpack some of these very issues. And I will first give the floor to Ambrish, who has a PowerPoint, which I will show you in a second. But Ambrish, you have the.

Thank you. Yeah, I can do without PowerPoint, if that helps. But I can start. Thank you so much for inviting me to this panel. I think it's great for us to be part of this initiative on security lexicon is something we deal with on a bread and butter basis. and would love to contribute. So just to set the context, right, I think as you mentioned, a security lexicon is something which should be a shareable language across multiple communities. We also play with words in the technical community and try and give a formal definition when we can. But the world is changing with this technology, and I think it's worth shedding light on what that entails. So next slide, please. So To begin, I think most conversations on what safety and security mean for AI typically begin with the question, what can go wrong? And this could be things from hallucinations to attacks on AI models to biases and fairness questions. But as soon as you start deploying an AI system, your lens changes a bit. You start asking question of what my current system is. what is it that it lacks or what are my control points? And how can I prevent harm from propagating through the system? Once you have one system solved, you start thinking about more interconnected systems and you move to a much broader question, which is how does the security properties change across an interconnected web of systems? How does this affect the economics of security? Who has access to what tools? And then finally, once you have looked at these lenses, the much more broader and perhaps a much more important question is what do the communities eventually trust? And I think a useful lexicon is one which can help think through the mechanisms, which are the underlying mechanisms behind any observed behavior. And it can think through what are the consequences of control. And I'll get into control a bit that the ambassador mentioned. But that is probably what the definition of a useful lexicon is. Next slide, please. Taxonomies and lexicons have been looked at by multiple different groups within the technical community. And I'll highlight five of them. They all look at the same system from different lenses and ask different questions and introduce different terminology, and you might often find some overlap between them. So think of the first one is risk and harms. The central question there is what are the constituent harms? What is the set? This could be, as I mentioned, biases, hallucinations, jailbreaks, prompt injections, some of the terms you might have heard across the literature floating around AI, even what AI is to begin with, right? And this sort of lexicon is useful, the risk and harm taxonomy in particular, for governance structures. The second, is to look at the threat model aspect. This is more of a security lens and what you're looking at is how could this system be attacked, be it an AI system or be just a software system which can be attacked by AI. And the type of questions you ask when you're doing a threat modeling exercise is what is the attacker's exposure? What is their access? What are their capabilities? What strategies can they use? And then eventually you want to rate a system so that you can preemptively fix some holes and develop defense in depth. These are the sort of questions you have in mind when you're looking at threat modeling landscape. And third and more important one in some sense is the safety and reliability, where you ask a question, when can a system fail? Under what circumstances can somebody leverage, even without a malicious actor, right? Like you can also have situations where a system just fails because of some inherent failure modes. then finally, when you're developing these technologies, a central question to ask is, what are their capabilities? And this sort of has become turbocharged with the use of AI, right? What are the resources required in terms of AI deployment to achieve a certain goal? Can AI autonomously achieve certain goals? What are the expertise required to deploy a system? So that's more of the capability lens. And then finally, I think an important one is the incident one. So if something has happened, Can we do a root cause analysis? Can we look at the control structure and make amends to it in order to preemptively defend better in the future? Next slide, please. So just to set some context for what AI terminology is typically from a technical standpoint, what the lay of the land is. So an AI model, which we talk about, there's a bit of animation there, so I might ask you a few more clicks. So AI model by itself is something that is, inherently stochastic, what we call it in the technical side, which is it has a variable behavior. It's not going to give you the same output every time. And that's something central to LLMs or AI models. They also work with natural language, which makes it difficult to separate instruction and data. When you ask it to send an e-mail and give it a body of text, you're doing two things simultaneously. You're giving it an instruction to send an e-mail and you're providing the body of the text, all collapsed in the same prompt. That is something that was not traditionally present in previous software systems. There was a much cleaner distinction between instruction and data. Next slide, please. But the AI model by itself sits within a much larger context and a much more sophisticated and layered architecture of software systems. You can think of this as in the medicine or the pharmaceutical domain. The medicine by itself has certain properties, but its overall effect depends on its dosage, its prescription, and how the patient is being monitored. In the same sense or a similar spirit, an AI model sits within a much sophisticated architecture where it has access to certain internal states of the software. It has access to certain tools that interact with the outside world. These could be e-mail tools, retrieval tools, information gathering tools. All of this run within what we call a containerized system typically, which is a runtime, which could be a cloud or microservices. And then to add to it, we have interconnected systems. So an AI system might be connected to other AI systems. Next slide, please. But within this, you may have situations where something undesired happens. For example, if somebody uses an AI system to summarize an e-mail or summarize some consultation documents, in the process, they might go through the documents and inadvertently send an e-mail because the body of the text said so. Now, in this situation, what we call typically as prompt injection, an unauthorized tool, e-mail being the one, was accessed because of certain instructions present in the body of what you were summarizing. Next slide, please. But this is not something that happens by itself. This is, as I mentioned, there are control points within the overall architecture where you could have prevented such a behavior. This could be input/output checks on what goes in and out of the AI systems. Typically, we refer to those as guardrails. You could have permissions and authorization and credential checks on tool access. And then even within the sort of containerized system where the software is running, you can have sandboxing and monitoring discipline. And this is what typically we mean when we say control in terms of software systems. Next slide, please. But having said that, the landscape has quite changed, right? That was a simplified view of what the architecture looks like, but the landscape has quite evolved. And I think when you're thinking through security lexicon, I think few things become important. One is the security exploitation of the AI based software itself. So when you're thinking of the applications that you're using day-to-day, can they be attacked? Then there is the second question of the malicious use of AI, what we call is AI for security or AI for harm, right? And then there are the broader security risks because of the interconnected systems. So I'll sort of break it down into four important themes which one needs to look at when inspecting this broader landscape. One is the development lens. So when we are building the AI technology, whether it is general use or because of its general use intentions, you may run into something called as dual use capability. So same set of technologies may be used by both attackers and defenders. So you may use or develop AI systems to do better search, better planning, But so will malicious actors. And in the development cycle, you need to focus on another key aspect, which is the testing of AI systems. So before they're released. Second aspect is the economics of this, right? So what does it take to make a general use system accessible? Is it a chat bot? Is it e-mail or a travel agent that you talk to? What is the cost behind hosting such a system? Can can an everyday person spin up something and host it? That brings to the third aspect, the distribution aspect. How is AI actually distributed across society? So there are two mechanisms which are of importance in the model world. One is this open versus closed models. Open models are ones who weights are these files essentially, is a downloadable file that you can download over the internet. Closed ones are ones which you just access behind an API, the ones that you chat about, you don't know about their internal states. But within the distribution, you need to also account for the organizational structures, right? This is something we've seen within airline and healthcare industries, that it's not just the piece of technology, it's how it is operationalized that is important. And finally, is the automation angle. What does it mean to have an automated system? Within the AI lens, that is something what we call long and short horizon tasks. Do you book a travel make a travel booking by going through an agent and let that chatbot handle all aspects of your travel, be it booking car rentals, hotels, et cetera, or does it come back to you? There's a human in the loop where it comes back to you, ask for validation. If it is done end-to-end autonomously, that's more of a long horizon task where you just leave it and it runs in the background. But if it comes back, there's a human in the loop, there's an authorization. There is a permission policy that determines when and when it cannot book certain things. And all of these, as I mentioned, are sort of turbocharged with the use of AI, because now you have AI generating more output at large volumes, and the whole system is interconnected, which means we have more work to do these checks. Next slide, please. So I'll conclude by sort of giving points in what the security lexicon going forward would need to address. I think a key aspect that it needs to address is it needs to help you arrive at decisions. It needs to simplify things for you. In this evolving world of more terminology, AI generating its own terminology, you need to simplify and arrive at decisions. And that involves four important buckets. One is, what is the system? What is the scope of the system? Who is being affected with the use of that system? Second is control and checks. From a technical standpoint or from an operationalization standpoint, can you set limits? Can you set limits in terms of amount of information you will allow within an organization to be dispatched or sent to an AI system? There are limits you might have seen, like five hour limits of not using AI beyond that. And being very clear about what are the interventions within a deployed AI system. And finally, once you have decided those checks, I think how do you enforce them? And how do you gather evidence for whether those enforcements were successful? And in the final bucket, you would have response and remedy. So what happens in case of failures? And even when you've done those checks and balances, is there remaining risk? I think thinking through these questions is important when it comes to thinking about AI lexicon for the future. I'll conclude with that and leave it, give it back to you.

Thank you so much. Thank you so much, Amrish. That was really brilliant. And thank you for outlining for us the many layers of thinking when it comes to security of AI systems, from software to deployment to post-deployment and all the layers we need to take into account, as well as providing us with some pointers on what a lexicon should achieve in order to be really an effective tool of governance. So now we will move to Jessica before we will open the floor, of course, for questions. And I invite you all to already start thinking about questions for our panelists and to really take most advantage of their expertise and presence here.

Jessica, you have the floor.

Great. Hi, thank you so much. Really appreciate the opportunity to share some remarks here. And I'm also grateful for Ambrish for doing the technical architecture talk, so I didn't have to put that slide up. So for some context, my name is Jessica G. I work at a policy research organization. So while I have a technical background, a lot of my research focuses on how to kind of think about technical developments in AI and primarily cybersecurity, which is another area I focus on and communicate those to policymakers. So my remarks are actually slightly different in character. I was thinking about kind of like three top level takeaways that I have that I have found in trying to think about some of these issues, talking to kind of like technical stakeholders in this space, and then how that kind of complexity can get translated to policymakers and decision makers. So the first point I wanted to touch on is that when we're thinking about how terminology develops in technical communities, I think this is also a, it's a social process. I think technical terminology develops over time. through both formal and informal channels. So formal channels being like academic conferences, academic publishing, convenings such as the ones we're having, or the one we're having right now, but also informal channels. So like in AI and, you know, machine learning, computer science more broadly, like terms that are now very commonly used can come up, you know, just from a blog post that a researcher published privately, and then that gets shared within a small community, and then over months and years, that gets adopted broadly until we're all using it, and it's difficult to trace back to the original source, even though you can point to, here's the exact first time this term was used. So it's very social, and I think it's also quite context-dependent. So in this case, I'll touch on some research that some colleagues and I were doing on AI red teaming. Which is, so for those who aren't familiar, it's a very common kind of like testing procedure used to evaluate AI systems. It's a kind of, I would say, primarily like a qualitative testing methodology, and it's very context dependent, primarily using like adversarial testing techniques to probe for flaws or vulnerabilities or weaknesses in an AI system. And we did a study where we talked to practitioners from across the ecosystem. So people who are developing AI systems, people from cybersecurity companies who are testing AI systems, and then people who are concerned about just how AI was being used in their specific context. And we found that while they agreed on the terms broadly, everyone identified as an AI red teamer, everyone said they were doing AI red teaming, there was quite a lot of variation and differences in interpretation in actual practice. Like even if everyone is using the same term in this technical community, we found that not everyone, when it comes down to the nuts and bolts of it, the more detailed you get and the closer to the actual system you get, disagreements started to emerge. And a lot of this has to do with how specific the contexts are. So every company will be different, every country's system will be different, every military deployment will be different. And I think the differences that really matter are often differences of context. And so the second point I wanted to touch on is that the lexicon discussion will probably be informed by outstanding disagreements between key stakeholder groups when it comes to AI security, national security. So I'm most familiar with kind of the AI and cybersecurity discussion, and I think, you know, even though it's been a few years, Oftentimes, people from the cybersecurity community and the AI development community will still speak past each other on what the priorities are and what they think the key intervention should be. And I think it's really important to include members from these different stakeholder groups in the conversation. And this is not even to say, I think even if we are just looking in a national security context at within one military, for example, oftentimes, Like generals and people like technicians on the ground will have very different conceptions of how the same technology is being used or like should be used for like a single purpose. So I think these disagreements that occur like between communities and also like from people who are closer to the technology itself versus people who are making strategy are often very meaningful. And I think it is important to make sure that different stakeholder groups are being represented in this conversation especially at an international level, which brings me to my third point, which is that when it comes to AI, a lot of terminology is particularly value-laden, and some of it comes from the way these technologies are developed within very small communities, I think, that are that have strong geographical centers. So there's the, I think most people are familiar with how the US AI community is centered in California, and there's also, it's the same in other places around the world, like for example, in Beijing and Hangzhou. In China, you have very small communities of developers. A lot of these people know each other. They're also social communities, and this is where terminology comes from. It's from this social network and people working together. And a lot of the times, the terminology reflects a specific set of beliefs about like what the technology was intended for and how it was constructed. So going back to this discussion we were having about autonomy and agency, I think it's interesting that several of us have already brought up autonomy in this discussion. And it reflects kind of a belief about what we think these systems should do. So one way to think about AI agents and risk from AI agents is that kind of autonomy is equivalent to intentionality. Like when the system is acting autonomously, there is some intention behind it. Like either the system is developing its own intention. And I think people from the AI safety community, when you think about kind of when people are talking about like autonomous hacking and things like that, they will often turn to this question of alignment. Is the intention of the AI system the same as the intention of the people who are making it or who are running it? But another way to think about autonomy is less about intentionality, more about how the system is constructed. An AI agent can just be an AI model that's given additional tools, privileges, and what we call a harness, which is just like software that enables it to do more stuff. And so some security practitioners have said, oh, the intervention here to make these systems more is just to remove some of this, the harnesses, or make sure that it doesn't have all of the privileges that would let it do these things and let it get out of human control. And I think these are different lenses to think about not just autonomy, but framing this discussion around what is safe, what is secure. And again, to reiterate the point that people from different communities will bring different perspectives, on what priorities are and what meaningful intervention should be in this space. And I think in Ambrish's remarks, I won't reiterate these, but he's already brought up a lot of terminology that I think is kind of like, I would call like value-laden. So human in the loop, for example, these are important questions like where should the human be, what is the purpose of having the human in the loop, like what intervention is the human expected to perform? on an AI system. So I think these are these are discussions in this debate should be included in the lexicon, like regardless of whether or not there is like a shared point of view. I think disagreement here is also will also be quite meaningful. And I just wanted to conclude by saying, like in this discussion, I think it's also important to avoid what the artist Eric Salvaggio calls the system from nowhere fallacy, which is the idea that AI systems are kind of just they just like appear in the world and begin to act on their own and aren't really, and humans just have to kind of like deal with their presence and their actions. I think the idea that like these systems and this vocabulary that we're all using, it's constructed by people who are operating under a specific set of incentives and belief systems, I think is important to note, especially when, you know, we may or may not agree with these points of view and where we see this technology developing, yeah, thank you.

Brilliant. Thank you so much, Jessica. I invite you all already to start thinking about the questions. And I want to say before that, Jessica, that I appreciate it also. I want to pick on one of the last points you mentioned on disagreement and highlighting in a future lexicon that we will develop also areas of disagreement, because we do not expect, of course, that all AI terminology will just We will just find sort of consensus across communities and perhaps in its very first iteration, the first volume of the lexicon, it will be more a collection of disagreements before we before we find the we we find more areas of of of consensus. Thank you also for highlighting interestingly the social aspects of of terminology of of AI concepts. I think when we think about things like like bias, for example, or other key terms that we use, we hear a lot. We just assume that these concepts exist as technical sort of mainstays that we cannot, that are just there, that just define somehow AI, but everything is, all of these concepts emerged in particular technical, in particular social contexts, and they mean sometimes different things to different people, depending where you sit on the AI sort of continuum of in development or deployment or something else. So, yeah, thank you. Thank you so much for highlighting all these differences. Maybe just before I hand it to the audience, I have a question for both of you, which would be, how do you think we can future-proof, in a way, a lexicon? So, not necessarily future-proof, but how can we make sure that a lexicon that we've helped put together with the help of other research partners, stands not really the test of time, because of course we expect that technology will evolve, that we will need to revisit some of the concepts, some of the definitions, the clarifications, the terminology, but how can we still create something that has a kind of a long-term usability rather than being just a 2027 sort of valid output? Maybe Ambrish can go first.

Yeah, I think two things come to my mind when I think of future proofing, right? One is the idea of simplification. So you need to make sure that you're simplifying terms when you're developing a lexicon as opposed to being exhaustive and including every little aspect. And I think the second, which through the course of at least studying AI system, the one thing that I have realized is A lot of the time we have to focus on system level properties and move away from very specific innovations that go into, say, a model building, which often get covered and highlighted, as Jessica was mentioning, through certain niche sectors. But overall, focusing on system level properties is the other angle, which I think is important for future proofing.

Yeah, I agree with what Ambrish is saying about thinking about system level properties. I think the more also that we use these systems in the real world, like moving out of kind of the realm of theory and into practice, the more we can build a better sense of what these system level properties are across different contexts. But I think this is a difficult question. I think given how fast the field is moving, there will naturally be some like attrition And concepts will also change over time. It's not that a term itself will fall out of fashion, but the understanding of it may change over time. So I think it's just worth making sure that that is included in the lexicon as well, maybe how a term has shifted and developed. And someone just may need to check back every year or so and make sure that a shared understanding still exists.

Thank you very much. So I will open now the floor for questions. And I see the ambassador. Please.

Thank you, Ioana. And thank you again to UNIDIR for bringing this conversation. I think it's a very, very important conversation. As I was listening to everybody here today, something came to mind that I want to share with all of you. One thing that Spanish as a language is spoken in different countries and that in 23 countries, we all understand the same thing with the same term. And that is not so. So when we realized this, all 23 of us, the Royal Academy of the Language in Spain and other academies of the language in all 23 countries got together and very painstakingly made-up a dictionary And that dictionary, sort of a lexicon from A to Z, reflects in each word what these words are understood or their meaning in the different countries. And this dictionary is updated every so many years, introducing new words and introducing new ways of understanding that word in Peru, in Bolivia, and in Spain. So as everybody was talking here, my question goes exactly there. Are we talking, when we talk about a lexicon, when we start this project, about a list in alphabetical order of what we understand by words that have to do with AI in D, or AI in a security context? Or are we talking about a lexicon? And this, of course, would mean that one would have to explore what the same word means in different cultures, because Jessica was referring to the culture part of this, which I find very fascinating and very true. And the way that words and concepts reflect how you constructed that AI and the how, how it was brought about. It wasn't brought about probably the same way in China than in some other country. So that was the my that is my first question. Or are we talking about a lexicon that has strict structured references and that establishes a common vocabulary, that establishes whether a use of terms is consistent, that is distinguishes between established concepts I mean, there's so many ways to do this lexicon. So, so the here goes my first difficult question. I'm sorry to the panelists and and that's it.

Thank you. Thank you very much, Ambassador. Are there any other questions as an immediate follow-up? Yes, please. And we'll answer maybe after we take two or three.

Thank you very much. And jumping off from the Ambassador's comment, I feel like that the very first word we need to define is to have a common understanding of the word security, because on one hand, We have heard about security of systems, infrastructure and frameworks, but on the other hand, as the lexicon is a product of a disarmament process. So a lot of the stakeholders are diplomats and military who use the word security differently in the context of international peace and security, national security, military security. So my question is, I mean, how are we supposed to differentiate or find a separate definition of the word security. Those who have handled the nuclear file know that the word nuclear security means very differently from the concept of security from nuclear weapons. So those are two very different terms. So I was wondering if we can apply a similar approach when we talk about security of AI versus international security in the context of AI. So thank you very much.

Thank you. Is there any other immediate question? No? Okay. Yes, please.

Thank you, Yona. My name is Tom Kono. I'm a retired UN official, now a part of the UNIDEL. You mentioned the disarmament community, and also national security community, they are all different approach to the same concept. But as the ambassadors over, you know, the definition of nuclear weapons never been agreed upon. And nuclear weapon free zone. They are getting close to the agreement, but once the definition involves who decides what the General Assembly decides or declares, then there's no agreement among the people. So the same thing can be applied to AI weapon, space weapon, was a security, cybersecurity. What the cybersecurity really means in terms of defense, in terms of global governance. So how this lexicon contribute to this kind of differentiation in definition, conceptualization. And I really appreciate your socialization approach because Everything comes out of some ideas. If they agree on the same concept, it's easy. But if they have a different approach and a different concept, then lexicon could be more complex. But it's also useful to see the difference among the people who get involved. So just my one question I have is, how lexicon will facilitate the negotiation on a certain global governance arrangement? It's a big question, but if you could help a certain key words, whether it's security or certain words, if you draw attention to, it would be appreciated.

Thank you. We'll just take one more, then we'll answer, and of course, we can take more questions. Please.

Yeah, thank you. Pretty much in the same direction, I guess. Not too pessimistic, but maybe realistic. Lot of debate took part in the GGE on the definition of what human control actually means or what does entail. And my question to it would be, does the AI lexicon can really deliver a consensus-based, precise, technical, or legal definition of meaningful human control? Or in the other way, I'm looking towards the ambassador of the Netherlands, do we need consensus? same direction over there, on specific terms, or do we need a lexicon like in the Spanish language on the terms, what could it have, or does the lexicon is able to provide us a tendency how we describe meaning for human control, or other way around. If this is the case, then I think, no, coming back to the pessimistic view to it, it doesn't help too much in the discussion of the GG laws on technical terms. Sorry for being so honest.

Thank you. Is it, Yasmin, is your question a follow-up or...?

No, thank you. My name is Yasmin, I'm from UNIDIR as well. Just a quick, I guess, more of a reflection. I don't want to be that person who says, "Ah, it's more of a comment than a question." But I think what I've really found really interesting, building on today's conversations, but also on what we've heard in the room in terms of the usefulness of a lexicon and why it's not really, why it's called a lexicon in the first place. UNIDIR has a very successful outer space security lexicon that exists. that is available online. It's called out space security lexicon and it's available in all six UN languages. And what I really found that is really useful is that, and I guess it's also part of the, it's, there's the same spirit as well in the draft resolution on the, in the military domain is that the lexicon does not seek, and I think for drawing on the comment about the consensus, but also what what I've heard here, what could be useful is for the lexicon to be descriptive. It's really served more as an encyclopedia to understand what are the existing definitions and approaches that exist across different languages, across different countries, across different disciplines as well, bridging policy, law, and the technical side. So I just think it's, I just wanted to say I'm very excited about this endeavor because it's, I think that in the absence of consensus and in the absence of agreement, at least understanding where your counterpart is coming from and the starting point across different languages can be truly helpful, so to speak, to understand. And I know that, for example, at the US-China track two level discussions between Tsinghua University and Brookings Institution, One of the key things that they've said was that one of the biggest confidence building measures that they had was to draw a lexicon and not necessarily to, again, come up with agreements, but to really understand what is the starting point of the counterpart. So I'm really excited about this project. I just wanted to add, thank you.

Thank you so much, Yasmin. And I think it preempts some of the arguments I was going to make in response to all the questions, but by the way, excellent questions, I think they really touched on the depth and they go to the core of the problem in a way because, of course, developing a lexicon is not necessarily an easy endeavor, but it can be extremely helpful and can go a long way in really helping to clarify how different thinking, different concepts exist and are socialized across communities. And of course, our intention is to make this as a descriptive tool. It's not going to have one definition only and impose it as the one definition onto the whole world. So it will really highlight how different, across different research communities, certain key concepts are understood, how they are employed. So I think that can really help illuminate at least how different understandings exist on this concept. To the first question on how we want to do it, of course, I think Our methodology is still being developed, but the general direction we want to take is to have, to not necessarily go in an alphabetical order and just define concepts or define terms, but really look at brainstorm and taxonomize areas of risks across all the research communities that we will, and with the research partners that we will bring together to identify together taxonomies of key concepts in AI security. What are the main concepts, the main terms that are part of the security vocabulary? And then to go into this descriptive exercise where we will show where how different communities think about that concept, but also if there are some cases where there is more or less consensus, we will of course also highlight that that particular term is more sort of unanimously sort of understood to mean the same thing. Of course, we don't expect, we don't know yet how that process will unfold, so we cannot pre-assess or pre-define what the final, how the final structure will look like. Maybe there will be just more disagreements than convergences, and that's okay. We will just highlight that and we will just show that this is where we are at. The second question, very interesting also on AI security or AI in the context of international peace and security. I think that's we're not going to analyze necessarily the implications of AI for international peace and security. The lexicon is really a technical resource, which contains terms on security of AI as a technical system and everything that has to do with the integrity and robustness of AI systems. So we will unpack those concepts and we will highlight what those concepts mean, why they are important, and so on and so forth. So it's really more looking at the technology first and then, of course, based on that, different policy communities, different experts can take it and sort of use it in the context of governance, in conversations and so on. How a lexicon can help negotiations, I think in a way it's already more or less implicitly addressed, that it can show these areas of divergence and convergence, and this is valuable in itself. It can show how different communities think about the same topics. It can clarify where the nuances are, where the divergences are, and it can be a further step into into finding consensus or if consensus is not possible necessarily on all terms, it can still provide this level of clarity, a common reference point. And then also moving on from there to the discussion on the GGE laws, of course, an AI security lexicon is not only about the GGE laws. I mean, it's not meant to just support the GGE laws kinds of conversations, which of course are concerned with topics that are in some ways narrower, like control or reliability, or there is also something about bias. So these are just a few concepts. The lexicon is a broader resource that looks at AI, or it would contain terminology that is also relevant for AI in the context of cybersecurity and integrity of AI systems. and it can support more broad conversation, not just on the G20 laws, but I think even there it can be an extremely useful resource to help clarify a bit more and to go into a bit more technical depth without as much as possible the political infusion that we see a lot in the G20 laws process, of course. We want to highlight also more strictly on how different technical community, different different technical perspectives, think about the specific technical concepts. So, yes, just to kind of reiterate that this will absolutely be a descriptive tool, a descriptive resource, and it's not meant to be in any way kind of imposition. And then also to tie this with some other things that were said is it will be a living document in a way, because for a technology like AI and I guess for any technology in a way, It cannot be the end of the road, just having one version of the lexicon and it's finished. We will have to maintain a sense of openness to keep updating it, to keep thinking about how certain concepts evolve and to keep it relevant in real time or as much as possible. Are there any, yes, please.

Thank you so much. My name is Anton from Russian Permanent Mission in Vienna. I'm dealing with the AI issues in the IAEA and the CBT. And taking that fact, I can understand that today we were speaking more about if we can call them simple AI tools like chatbots, copilot, etc. But when it comes to the more technical fields, the more advanced AI systems would be used there. Do you take this into account while developing this lexicon vocab? And if I may, a second question. It's regarding your comment that the concept of your work is still in process. In that case, I thought about Jessica's very important remarks about invitation to the interested stakeholders to the process of the development of such kind of lexicon. In this case, my question is, if you don't completely understand the concept, how do you identify who exactly do you need to invite to the table as an expert? Thank you so much.

Thank you. Are there any other questions? Yes, please.

Yes, Sophia from the UK mission. I think I have more of like a curiosity-based question than anything else. How long do you think this is going to take you to compile? Because I assume it's quite an ambitious initiative. Do you think it's going to take you a couple of months, a couple of years? Do you have a time frame in mind at all? Thanks.

Hi, Bobby Wilson, a fellow at UNIDIR. So from my understanding, a lot of the lexicon, the goal here is to report the current meanings of the words that we're using. And bringing up the point that Jessica had made about the social constructs, I think this is more of just a comment as I'm thinking about that, being more on the technology side myself, that I'm not sure, I'm not convinced that people who are originating these words are thinking about the implications of the words they're using while they're doing it, because they're using it in a very specific context, and so they're not actively thinking about that. And I think that that can, it's something I think to just be aware of as these definitions are kind of being put out there. For example, the constant like anthropomorphization of these systems is something that's rapidly seen out in the world, but I think that providing a platform for that to propagate can be a real issue. for Unitir and just the dialogue at large, I think it can hurt the problem more than it helps. So I think just being aware of that as you're kind of moving forward. Thank you.

Thank you. So I'll just take a first stab at the questions. And by the way, Ambrish and Jessica, if you want to complement anything I want to say, of course, you're not part of the Unidire team, so I understand that you cannot answer some of the questions because you're not in the team develop or working on the lexicon, but if you have any reflections, please don't hesitate to interrupt, I mean, to tell me to give you the floor. So the first, the first question, and thank you for, for these questions on, you know, the, the simple, the, the more simpler parts of the technology and then the more complex ones, um, I think, so we all have these assumptions about AI, we all, we all… Every one of us in this room has assumptions about AI, even the scientists, I'm sure they have some assumptions that we just sort of go with. And I think it may be that's perhaps can be a dangerous place to start. If we just assume that some things are simple, some things are inherently very complex, and we should only include the simple ones or the complex ones. So I think we will start with by simply having first a sort of wider and in-depth sort of brainstorm exercise with many universities, with many technical experts, with many research institutes to find, to identify together what is worth of including in a lexicon and what are sort of perhaps very, very, very marginal or niche concepts that are not that interesting or useful for a wider community. So we will start this process, not necessarily with specific assumptions. Of course, we have some directions and some sort of starting points, but the idea is to first do a taxonomization together with other research partners on what should go in the lexicon. And that also brings me to the question on who to invite, and it's a very good question because of course, depending on the kind of concepts that you want to explore, that might also shape the sort of expertise you want to bring in. But the first sort of layer of doing that is to just start with, have as a starting point the goal of inclusivity. So we would really want to include many, many research partners from all UN regions, from many countries, and to have a joint conversation, because we cannot make these calls early on and be sort of, or limit in a way preemptively who is part of the conversation. So, yes.

Thanks so much for this explanation. Just to add to my second question, Do you now believe that at some point, governments should be as a part of this conversation?

No, excellent. Excellent, excellent question. And I think it was already sort of raised and discussed even in maybe some of the conversations in, you know, on the resolution. This is a research led process. We are thinking of having a sort of maybe stress testing of how the concept or a final version or a pre-final version may be received. And by stress test means just to present it to governments in terms of, is this understandable enough? Is this clear enough? Just in terms of the political sort of, is it digestible to the policy audience? But the development itself of the lexicon will be scientific and technical driven rather than including states in the or government perspectives per se in the elaboration of the lexicon. But that said, of course, we want to Before we actually publish it, we want to stress test a bit the understandability of it and just to make sure that it's palatable to policy communities, but they will not be heavily involved as primary actors in the development. And on the timeline, a very good question, maybe we should have also mentioned that so that we clarify a bit on expectations, we want to do this by the end of next year, at least as a very first output with the understanding that, as I mentioned, it's kind of a living document, something that will need more, maybe updates or adjustments in time, but we want to to have something by the end of the year. We could also opt for, you know, the option of having it over three, five years because it's so complex and you can over engineer it and then, you know, you can have a, or you can take forever to make it, but at some point you also need to sort of have more specific and more well-defined targets so that you actually produce something. And I think also the goal, and I think it's taken me to something that Ambrish said about how to think about the concepts and how to future-proof it, is that while it's important to be very rigorous and to have a clear methodology and to do this in the best, most inclusive way, it's also important not to over-engineer it and to over-complexify it, because then it will also take not just longer, but it can also in a way be self self harming to the whole process because you don't really have a clear direction and you're going all over the place in a way. So I hope that clarifies it. But yes, the goal is to have it maybe by the next first, the first committee. So we actually launch it. That would be ideal. But if not by the end of by the end of next year, we are we are at time, but we Yes, yes, Ambrish, please. Of course, of course, Ambrish.

Sure. Quick remarks. One thing that, listening to the discussion that comes to my mind, right? So typically when we think of lexicon, I think one of the purpose of the lexicon is to actually enlist the purpose of the lexicon, right? So one is to simplify understanding and one is to communicate. Second, I guess the third one is to think through priorities. And finally, I think to take some decision, right? These are eventually the use, eventually where a lexicon would be used. And I think given the context that we're talking about AI, I think one needs to be very cognizant of things that have been brought up, that there are few things that emerge out of the tech world or other social subcultures that eventually become common language, something similar to how a slang or internet language develops, right? Now, in that context, I think we just need to be very aware that most of the definitions would be working definitions. I think the notion of working definitions and the notion of things being contextual is just necessary as opposed to thinking of this as an Oxford dictionary of enlisted terms. I think these are contextual terms to begin with and they will evolve. So making that the first class citizen of any sort of discussion or discourse on AI, I think is necessary. Just wanted to highlight that.

Yes, thank you, Amrish. Indeed, it's a It's very useful to really, really emphasize on this point that we're not developing the Oxford Dictionary of AI Security as a, you know, as a fixed and sort of finite product. But we are working, we are we're showing evolving and working definitions and and the contextual part of it also. I saw there was, I think, a question on this side or did you have a no. Sorry.

Quick one, because I know we're, um, out of time, just a basic question of what would success look like, if you have any signal that would tell you that this has been useful for policy, um, thank you.

That's a, that's a great question, I think one, one of our, um… One of our success indicators is always if states refer to or if states actively say or mention our research as having been useful for them in public fora. So as simple as that, just sort of a recognition or acknowledgement by states that we took this and we found it very useful. It helped clarify concepts for us. It helped us make more sense of certain understandings or how others are thinking about certain concepts. So think it's just this kind of public acknowledgement by states that they looked at this and they found it useful and it contributed in some way to their national or international discussions.

And with this, I think we are actually already at time, but I saw there is, I think Ambassador Endemboch would like to add something in closing, and then we are at time, thank you.

Thank you. And for me also very enlightening. And yes, thanks also for confirming some of the things I thought that you were doing, but now indeed getting a confirmation. This is indeed not a vocabulary. This is perhaps also not an encyclopedia, but because you try to go a next step in trying to build something. Looking back at the work we did in the GGE, we started at the beginning with having the views of high contracting parties and then making a compendium out of it. And that was only a compendium. So in fact, what we did was just staple it and send it around. But I think it already helped in having just those views from a substantive number of key players on what they thought that, for instance, human control should be and what they thought that a lethal autonomous weapon is and what they thought that lethality is. And then indeed, I think very wise words in one of our first meetings from the Chinese ambassador was, yes, but we should indeed have a working definition and over time this can develop. For instance, once we start writing a legally binding instrument or whatever, then you might need to refine it again. And I think this is also what we will probably see with your document there. Somebody already said, kind of a living document, future proof, that is a challenge. And it remains a challenge because let me just tell you a little anecdote from the final night of our negotiations when we had a little phrase, rules of international law applicable in armed conflict. And then a distinguished representative of one of the high contracting parties, also as a mother language, having one of the six official languages said, "Rules is difficult." If I have to translate it in my own language, it will be difficult. Well, that is the challenge we are facing and this is indeed, we are looking very much forward to your work and yes, it will not be the end to the discussion, but it might be the end of the beginning of the discussion and at least having a something of a common point of departure for further work and for further discussions we're going to have. And also in the framework of the GGE, I understand that quite a number of UN member states, high contracting parties, have already proposed that there need to be further discussions, especially on text we were not able to agree to have the experts around the table and try to define what we should understand under certain words that were not agreeable in September. And I think having those discussions that might work, might also influence your work, but that has, of course, a certain time limit. And yeah, I won't be here next year, but we're always looking forward to having your results in for the next round of discussions.

Thank you so much, Ambassador. Thank you so much. You provided the closing remarks in a way without me having asked you before to prepare them, but it was very nice and very eloquent, as always, in a way, providing a general framing for what is happening and how this will be done going forward. Thank you all so much for your time, for your interest in this topic, in this event. This is the beginning, the end of the beginning, indeed. So we're just now launching this project and we've announced what we want to do. But with this, we are now going to start the actual work and we very much look forward to it and we hope that you will find it useful and interesting once it's out. But for more on that later. Thank you all so much and wishing you all a very nice rest of the day. Thank you.