9th Meeting - 1st Session Group of Governmental Experts on Lethal Autonomous Weapons Systems 2025
Meetings & Events
Date: 7 March 2025
Language: English
Transcript: https://transcripts.un.org/en/asset/k11/k11qdcu4id
Transcripts available through this tool are created by using automatic speech recognition and are not official records nor official documents of the United Nations. Official records and official documents are available on the Official Document System of the United Nations.

---

Chair [0:05]:

Yes, good morning again. Excellencies, distinguished delegates, I would like to resume our discussions again. I love it if you have informal discussions, but then I expect also agreed text from you. Otherwise, we start the meeting. Yes, please. Yesterday at the end of the afternoon we had quite an interesting discussion on some guiding questions for my part. I then said that some of you might still feel a need to reflect on them. And for that reason, I will now open again the floor with regard to those questions for delegations who would like to share their thoughts. And once we have done that, then I would like to move on with regard to the text of the updated box 3 that was circulated at the end of the afternoon yesterday. So you had some time to reflect on that also. But let me first switch back to the questions and see if there is still a delegation that would like to reflect on that. And if not, then we move on to box three. Let me open the floor. Yes, I see the flag from Germany. Let me pass the floor to the distinguished delegate from the German delegation. Ambassador, you have the floor.


Germany · Ambassador [2:07]:

Thank you, Mr. Chair, and thank you for the new text you circulated yesterday evening with us. I understand that you gave us a chance to come back to the questions you put before us yesterday. And indeed, as you did your homework in producing this new text for us, my delegation also was trying to do its homework and just to elaborate a little bit on on the questions and especially also on this element of human control, which was at the core of these. So perhaps at the outset, I would like to say that we really see a lot of of value in these exchanges this day and also the very skillful contributions we've heard already yesterday from the delegations here in the room. So let me just add some ideas from our view. So in Germany's view, a framework of human control is necessary to ensure compliance with international human law. And this implies not only technical control, but also an element of judgment. So, the framework of human control, as we outline it in our internal discussions, is a construct that encompasses the technologically possible actions by setting clear boundaries. within which the system's algorithm is allowed to operate. International law, and in particular, international humanitarian law, is one of these boundaries. So these measures, with particular attention to target selection and other measures to which I will refer later on, from the assessment and control part of the process and are reviewed on an ongoing basis. So for the operation of such weapon systems featuring autonomy, the activation process is ordered by a responsible chain of command and the operator is able to exercise control over the system at this stage. Furthermore, the framework of human control also requires appropriate rules of engagement to be developed and programmed into the mission software and transferred to the system in the activation process. So the rules of engagement address the issue of judgment by integrating the legal aspects and form the conditions under which the system can operate in the theater. They are also constrained by time, geography, and the range of targets it is allowed to engage. As part of the wider framework of human control throughout the life cycle, we pay special attention to the training of the personnel who operate the system. The testing of these weapon systems in accordance with Article 36 of Additional Protocol I of the Geneva Convention is also a building block in the framework of human control and ensures that the weapon system can be used in accordance with international law, including IHL. So the framework of human control is not limited to the moment of the actual use of the weapon system, but is also necessary during its entire life cycle. So in Germany's view, therefore, direct control is not always necessary to comply with IHL. It can and mostly will be exercised before the selection or engagement part. Thus, because control of algorithms and operations is exercised by human, by humans, human judgment is applied within the responsible chain of command. So this would be our understanding of context appropriate human control and judgment in order to ensure throughout the whole life cycle and for military operations in order to ensure compliance with IHL, particularly the principles of distinction, proportionality, and precaution in attack. The critical functions of the system, so identification, selection, engagement of a target, will then be performed autonomously on the basis of these mission parameters and authorizations upon activation and with no or only limited further direct human control. This underscores that the activation moment is a very critical moment and it's therefore we have pointed this out several times throughout our interventions. So that would be in some the reaction, the comments we would like to make with regard to the questions you put before us yesterday, and we will then come back later to comment more in detail on the new text on box 3. So, thank you very much, Mr. Chairman.


Chair [8:11]:

Many thanks to Ambassador Goebel for sharing the views of his delegation, in particular with regard to how Germany sees the working of context-appropriate human control and judgment. Let me now pass the floor to the U.S. Sir, you have the floor.


United States of America [8:34]:

Thank you, Chair, for giving our delegation the floor. and for your leadership this week. I think we wanted to come back to the conversations that we began yesterday in the context of your questions, but I think the thoughts that we're sharing this morning will also animate our views as we work through box three and if there are further discussions of additional boxes over the course of the day. As the distinguished ambassador of Germany just noted, we think there is great value in this conversation. And in the spirit of interactivity, our delegation wanted to respond to a number of the points we heard in the conversation yesterday, which in our view was quite revealing. Yesterday, you got delegations off of their scripts. You asked them to say what human control means. And I think when we've had these interactive discussions and we've been able to get delegations off of their scripts, it's yielded some of the richest and most important discussions that we've had in this forum. First point, we heard a fair number of delegations yesterday say that human control is required, but this does not actually mean that the human must control the weapon during its operation. If the group negotiates a formula of words, and the first thing we have to say when explaining it to others is that it actually doesn't mean what it sounds like it means, then from our delegation's perspective, we've chosen the wrong words. Mr. Chairman, we're negotiating language about what international law requires. The ordinary meaning of the words we choose matters. If we say one thing and mean another, we risk confusing and not clarifying legal requirements. This is our second point. Mr. Chairman, the advocates for meaningful human control or Context-appropriate human control say that it is required by international law. This is what you have proposed in your rolling text. Human control is what is essential for compliance with international law. And just to anticipate the discussion we're having about box three today, you have also proposed framing requirements as flowing from this purported rule. What did those who support this standard say this requires in our conversation this week? Well, it depends. It depends on factors. It depends on context. Did anyone say clearly what was prohibited? What is always required? Well, it doesn't actually mean a human has to control the system directly during its operation. The fact is, Mr. Chairman, that there are almost as many interpretations of this phrase as there are delegations in this room. At least one delegation has said that human control means whatever my country says it means. Listen to that. They're essentially saying that what is essential in order to comply with international law means whatever my country says it means. In fairness, they want to give every country in this room the same discretion. But this will not promote a unified understanding of the legal framework governing laws or the measures necessary to implement them. This is not what we want. Is this what you want? Mr. Chair, we are negotiating and drafting the elements of an instrument. The words matter. If we say one thing but mean something else, we undermine the credibility and effectiveness of the instrument and of the multilateral institutions and processes that produced it. If we say that meaningful human control, what is essential to fulfill the requirements of international law, mean whatever we think it means, or that it all depends on the circumstances, and cannot be reduced to a rule, then we make a meaningless instrument that undermines existing rules of international law. If the law can mean anything, then it requires nothing. Mr. Chairman, my third point that we need to recognize is that we need to recognize the reality of human judgment and control in war. My delegation heard yesterday that all existing weapons have control built in. It is these new weapons with autonomous functions and features that pose challenges for control and accountability. They involve the delegation of life and death decision making to machines. The reality of war to us is far different. War is chaos and accidents. and many severe consequences outside the best efforts of humans to exercise judgment and control over the use of force. Every weapon has a point at which its effect are outside human control. You shoot a gun, you cannot control the bullet after it leaves the chamber. You fire an unguided rocket, it cannot be controlled after firing. The use of normal human control weapons in combat with human control and judgment often involves unintended effects and human error. The practical effect of autonomous weapons-- sorry, the practical effect of autonomous functions in weapon systems is that they can increase control over the use of force. They can be used to create smart weapons that can be used with greater precision and less risk to civilians and civilian objects. Mr. Chairman, there is nothing unethical or illegal about this. If a computer can be used to make a decision on a battlefield that saves lives, My delegation believes it is more ethical and more consistent with the law to use that computer rather than to let civilians die, because at least then decisions about life and death won't be given to machines. Mr. Chairman, I'm making a pointed intervention here. But these points are offered in the spirit of genuine and frank dialogue. These points are offered in a constructive spirit, in the spirit of pointing us in a positive direction. We need to say what we mean. We need to be clear and specific in giving guidance to states to implement. We need to reaffirm and clarify the framework of existing IHL. We need to recognize that the very technology at issue, autonomous functions and weapon systems, can be used to save lives and strengthen the protection of civilians. We mustn't privilege the dumb weapons of the past over the smart ones of the future. Thank you.


Chair [15:46]:

Let me thank the distinguished representative of the United States for sharing his views. Now, I have no problem at all with a pointed intervention because I do hope it will trigger other interventions because, indeed, we are at the moment in the discussion where we We should really have clarity about what we want and what we can achieve. And yes, words, of course, do matter. I agree with that. We are not trying to make an empty text. So thank you for that. And thank you also for your views. There are more views around the table, I see. That's good. Next on my list is the distinguished representative of Austria, to be followed by Article 36, but first, Austria.


Austria [16:41]:

Very much. We just wanted to react immediately because it is a follow-up of the discussion yesterday and maybe also some things that we said yesterday and how we kind of reacted to your question which was off script and maybe we said some things that could be understood in a wrong way. But on the other hand, it is probably how delegations feel about this question, how they feel about human control. And aside from our own intervention, we heard very excellent interventions, as the one today from Germany on these questions and what human control means and what it is there for. So the questions definitely did serve their purpose, and we are very happy about this discussion. We just want to, I mean, we are now recapping the discussion from yesterday or the one that follows today, and we just wanted to react to the reframing or the interpretation of what has been discussed or what we and other delegations have said. I think the first point that I wanted to react is about how human control is understood as a word and that the meaning that control generally implies is not what delegations or what we have implied yesterday what it does with relation to autonomous weapon system. I would see it, if you are going into the semantics in a different way, that we have to see human control in relation to autonomous weapon systems. Because intuitively, persons will also think that there should be human control over weapons. that there should be human control over autonomous weapon systems, and this is the spirit in which we read many of the interventions. If we would go into a direction where there is no human control over autonomous weapon systems, if it's just human judgment or averted means even less, we would go into a direction of fully autonomous weapon systems. And this is something that many delegations would see as something they don't want, or that is highly undesirable, or that is even militarily absolutely useless, because which military wants to lose control over their weapons? So control in general is needed when it comes to weapon systems. We had this in the 2023 report. And the human element is one of the key assets that we are having in this discussion and something we want to maintain. Some formulation that we had in one of our working papers that I yesterday thought was useful, but I did not bring it up, was that it is absolutely essential that everything that happens on the battlefield, especially when it comes to the use of force, can be traced back to human agency. What we are discussing currently is how this is done. And for most delegations, as far as I can read the room, human control is one of the words. What we have done yesterday is splitting up the word maybe, and this was also what led to my two interventions into different levels or different ways that there is human control on a global or general level, something that has to be achieved. And we are very happy that this is also reflected in the box now. And then the specific measures on how to do it and how it is actually, if, if effectuated, and maybe this also creates to the, to the misunderstandings that there are two levels to human control, especially when it comes to the semantic dimension. It's something that you want to achieve, but it's also something very specific that you're doing during the operation. If you only focus on the second, we are coming into this discussion and into these problems. But there is also-- I hesitate to call it the normative aspect, but there is certainly something that relates to IHL in general and the overall aims. And in our view, human control over weapon systems is needed to uphold IHL, mostly because IHL is made for humans. And what we are actually doing here is to translate this. And this is absolutely necessary, that we translate how IHL can be used with the specific weapon systems and the new technologies that we are seeing. Second point-- and I know the first point was long. The second point will be shorter. the comments that we made about the circumstances and that this now created kind of very unruly interpretation of what we mean and how human control should be effectuated, this does not relate to the word human control. It relates to the word context appropriate. And context appropriate in our current discussion We would prefer meaningful. I think that's clear. But with the new proposed formulation by the chair about the context appropriate, this gives states leeway. And this is not per se, and it's strange hearing this from our side, a negative thing. Because the context appropriateness provides for the possibilities to adapt to different situations, to different weapon systems. and insert the human control that is needed. So the combination of the word context appropriate and human control is useful for states, but at the same time it gives it also some more meaning and not, and here we have to disagree, it does not create a you do whatever you want approach. Many international instruments in disarmament are not too specific on certain ways on how to implement them. Sometimes this is a good thing, sometimes it's a bad thing. But it is not that we are here in-- I would disagree with the interpretation that this creates confusion or is against the spirit of our discussions here. And these are just my two points. Thank you.


Chair [22:42]:

Let me thank the distinguished representative of Austria for sharing his views and adding to what he already said yesterday. Next on my list is the distinguished representative of Article 36. You have the floor.


Article 36 [22:58]:

Thank you, Chair. I think in your comments earlier, you suggested that perhaps the US intervention might trigger further comments, and I can confess feeling a little bit triggered. thought I would perhaps try to describe a little bit feelings about where we've got to in this conversation, thinking about human control and how we orientate ourselves to autonomous weapon systems. I just want to say, first of all, I think we should not lose sight of the fact that we've come a long way in these conversations. And it feels like as a group, although we, of course, are not all in agreement with each other by any means, it does feel like this conversation is now orientating around the subject matter in a way that there is a much clearer sense of shared understanding of the issues at stake and of the technologies that we're working around. And I think it's impressive that we've come this far. Just in terms of how we think of autonomous weapons systems, I think we think of them fundamentally as systems that use sensors to evaluate the environment in which they're operating. And under certain conditions, under certain inputs to those sensors, these systems are going to apply force to objects in the environment. For us, we would see this as being driven by the target profiles that those systems use. What pattern of sensor data is it that triggers the application of force by those systems? Target profiles that systems use are imperfect representations of the world. Machines don't see the world in the same way that humans do. They need to be programmed in some way to model the objects that they're to be used against. And we see, of course, target profiles in the form of the heat shapes of objects, the radar signatures of objects, the acoustic signatures of objects. These could all be used to direct force quite specifically to targets, but they're always imperfect representations of the world as well. And there's always some uncertainty as to whether they match the intended objects of attack. We see these systems then being used within some window of time and space. There's a period of time and there's a spatial area within which this system is functioning. autonomously and where it's dependent upon the inputs of those sensors. And for us, I think the longer the duration of operation, the wider the geographical area within which it's operating, the more engagements, the more sort of iterations of force that the system can engage in, With all of these factors, there is a growing risk that the assumptions of the operator are no longer bearing out in the way that the outcomes and the effects of the system occur in reality. Human operators need to evaluate the context within which these systems are going to be used. They need to evaluate the civilian objects in those areas that may trigger applications of force inadvertently, or civilian objects and persons that may incur harm as a result of collateral damage, for example. Nature of the context. can determine the extent of complexity in a way. More complicated environments with more variations of objects and phenomena within them may make it more difficult for commanders to evaluate and anticipate what the outcomes are going to be. And I think this is where the context appropriate formulation evokes that sense that in certain environments it may be easier to use systems predictably than in others. I think This is where judgment and control come in, you need to be able to evaluate these, understanding the system, understanding the context, coming to an informed judgment about what the likely outcomes will be. I think as with IHL, many other IHL rules, there's a degree of flexibility in the formulations that have been discussed here, which is probably a useful thing. But at the same time, if you can't limit the duration of a system's functioning, if you can't constrain its scope, if you can't adequately understand the types of targets that it will exert force against, then it's not going to be possible to exert that judgment and control in an appropriate in an appropriate way. So it feels as though in orientating ourselves here to a broad range of technologies, the types of rules that have been discussed with a broad framing of a requirement for judgment and control, coupled with specific parameters that need to be limited or controlled within that, seems to provide a practical orientation to regulating new technologies as they may develop in the future. Thank you, Chair.


Chair [28:00]:

Let me thank the distinguished representative of Article 36 to also reflect on the questions put forward by the Chair and interventions from other delegations. I'm looking at my list to whether I have, yes, I see, I recognise, yeah, I recognise the distinguished representative of Pakistan. So we have.


Pakistan [28:25]:

Thank you, Chair. Good morning, dear colleagues. Chair, we were prepared to, of course, go directly into the box 3, but availing your invitation and the interventions that have been made this morning, we would also like to briefly respond and react to some of the comments that have been made. And we believe that that, of course, would facilitate how we approach box 3 as well and the ideas that have now been introduced in the text. So we think that the exercise that we are involved in is geared towards that end. We have listened very carefully to the comments and interventions made by our colleagues this morning, and we found, I think, the intervention made by the delegation of Germany very helpful, and we We agree with many of the things that were listed in that intervention. We also listened very carefully to the thoughtful intervention by my U.S. colleague. We appreciate the clarity and the directness with which these concerns have been articulated. And I agree with him that the richness of our conversation is precisely flowing from these questions and the conversations that we have been having since yesterday. We continue to believe that human judgment without effective human control risks becoming abstract and operationally ineffective. We just heard that human control is perhaps not necessarily, it's not required during the weapons operational phase. I think the conversation that we had yesterday and where we tried explaining to other colleagues We believe that human control is required and was always present during operational phase, but it could be manifested in different ways. It not necessarily translates through a single action during the selection or engagement of a target. And that different manifestations of human control, of course, varies from weapon system to weapon system and the context as well. And it is precisely for this nuanced reality, the language context appropriate was introduced in the first place and was explained yesterday by many colleagues. The term in our view explicitly recognizes this variance and it recognizes that in some operational context, direct human oversight in real time is needed for essential compliance with IHL. In other scenarios, that human control and judgment is effectively exercised through careful pre-operational constraints and detailed rules of engagement, rigorous training, and some other measures which we will discuss and which are listed in box three, for instance, under paragraph 6C, in which we will come to when we start discussions there. So We agree with our colleague from the US that the words we choose here matter significantly. And precisely because they matter, in our view, we must avoid a narrow reading of the words context appropriate human control and the IHL obligations itself. We believe that they provide deliberate, clear flexibility and alignment with IHL, existing IHL principles. And it gives that flexibility to states for operationally implementing this guiding principle in the national context in the first place. Thank you.


Chair [32:16]:

Let me thank the distinguished representative of Pakistan for adding to the discussion. I'm looking on my list. I do not recognize any further requests for the floor. But I think it's wise to move on to box 3. Some of you have already referred to box 3 as that some of the remarks made are in kind of an introduction on the positions that they will take during our discussion on box 3. So I think this kind of a bridge is very useful. Let me again thank you for this off script, I think it's important, uh, as was mentioned by, by one of the distinguished delegates, um, this, this discussion has contributed, uh, indeed to, to, to, to have, um, a shared, um, or at least an increased, I don't know whether we already share certain, we might still have difference of opinion on how exactly, but the understanding with regard to the new technologies has certain benefited, I think, from this discussion. And it also stimulated, I think, further thinking about what certain concepts would mean. And I think we all have been challenged to do further work on that. I will look into the possibilities of how to facilitate that in the near future during the intercessional program also. But I will address that later today. Now with regard to box 3, we have the text already up. I would say that during our deliberations on box 3, across the room, there was support for the overall direction of box 3 and for the fact that it contains elements that could outline key building blocks for a future set of elements of an instrument in the form of prohibitions and regulations. And some of you have even suggested already titles for the text. And I felt that delegations are overall comfortable to build on existing agreed wording but we also have to take into account the ambitions of our mandate. Many delegations support what is in the text as appropriate prohibitions, limits and control measures. I support the structure and logic as well as the granularity of the text. I would also like to thank you in advance for providing explanations with regard to any proposals you might want to make to the text. I already said, explain why to retain or not to retain, as I would like to call it. This is very helpful. I have somewhere here on my desk a 61-page document that is what you have produced so far. On the other, it's good news because you really went into detail. You provided clear reasons why you don't want to have something or you would like to have something, or why you have certain comments on text. But our great challenge is, of course, to reduce those 61 pages then to four pages of concrete elements. So this is a bit the concrete manifestation of the challenge we are all facing. We had a great discussion, but we also have to, at a certain moment, to narrow down certain issues. Let me now refer to the text. On the paragraphs 1 to 4, by many of you these were recognized as a reflection of fundamental existing IHL considerations and obligations, agreed language from guiding principles, or from previous GGE reports. Many delegations supported building on those fundamentals, adopting a forward-looking approach, building on the agreed language and working towards elements in the form of prohibitions that could form the basis of a future instrument. But we still have to keep in mind the full mandate, what we have, including measures, et cetera. For the paragraphs 1 and 2 in the original version of box 3, a proposal to merge them was made and found significant support. Many in the room shared the view that such a consolidation would not result in any loss of substance and at the same time add clarity and give guidance for its operationalization by combining the prohibition with a positive obligation. We have tried to do so, you see it in front of you. With regard to paragraph 3, we decided to refrain from making any changes for now. The language that we have on the screen is closely aligned with Article 51 of the Additional Protocol to the Geneva Convention, but also to CCW protocols, and that's relevant, of course, for those of you who are not a party to the AP I. With regard to Para 4, the text builds upon numerous proposals. This is one of the paras where I decided to introduce "develop" instead of "deploy" so that the text now reads "develop or use". Para 5, I think for many delegations, para 5 is highly relevant or highly contentious para, because it's, of course, a key element or notion, and we have to get that right. Like in box 2, para 5 includes the term "context-appropriate human control and judgment." And we had already quite a discussion on that yesterday afternoon, and that continued this morning. That was the bridge to what we are now doing. It's clear we need more discussion. I think we have a number, we had a number of delegations already explaining either why context appropriate human control and judgment works in their view, or why it doesn't work. Since this discussion clearly has not come to an end, I have refrained from making any changes, and I think we have to continue our work, building on the discussion yesterday and on the discussion still to follow today. Now moving to the second-half of the text, the para 6 and 7. This is still affected by, of course, what is above it, the discussion on context-appropriate human control and judgment. Some of you pointed out, and as was done again also this morning, there's a lack of clarity and objective criteria to determine how it should be understood. Others made a link between context-appropriate control and the regulations that are listed in the second part of box 3. This is how the text is also structured. Nevertheless, to try and accommodate the different views, we decided to introduce some changes to paragraph 6, both in its chapeau and in its sub-paragraphs, to address the concerns voiced in the room. during the earlier discussions. In particular, I would like to point out that these changes to paragraph 6 could stimulate a debate on what the objective criteria to comply with IHL and context-appropriate human control are. What we have tried to do in the text is introduce a concept that the notion of human control is not necessarily tied to control over the actual operation of laws, but related to the effects of the use of laws. Some of you have also stated that. Others have different views, so we have to be aware of that. Subparagraph 6. We heard calls to create a general prohibition in the first part of box 3, being a prohibition of systems on the basis of the concepts predictability, reliability, traceability and explainability. On the other hand, some of you also proposed to move this language to box 4 and frame it as measures and guidance for national implementation. The revised formulation you see before you has opted for middle ground, which is to use this concept as a for the notion of context appropriate human control and judgment. And the changes aim at clarifying that the fulfillment of these criteria is a responsibility of those that use laws, for instance, the operator. Regarding sub-para 6, the changes introduced in this para aim at avoiding a circular reference to context-appropriate human control and judgment, which is now at the chapeau, while at the same time introducing the element of a responsible chain of command and control, which has been suggested by a number of delegations. This reference relates to the second part of this sub-paragraph, which is the reference to the need for consideration of target identification and engagement stages. We also took on board a proposal made by a number of delegations to replace the word "moral" with "ethical". On sub-paragraph 6, I have opted to avoid major modifications and choose to incorporate a few tweaks suggested by some of you with regard to the small Roman numerals I to VI, while at the same time changing the chapeau to clarify their standing as an indicative and non-exhaustive list of measures that could be used to ensure control over laws. The new Roman VI is included as a possible alternative to para 7 in order to address the issue of real-time machine learning. And both of them have been explicitly directed at the core functions of laws. Finally, on para 7, you will see that I incorporate more precision to what part of the mission parameters cannot be changed without context-appropriate human control and judgment. So far from my side. I now really would like to hear from you with regard to the changes we have made in the text and whether this is a way forward for a revised text to be circulated in due time after this meeting. I see -- is that correct, the flag of Germany? Then I would like to thank the distinguished representative of Germany for taking the floor and opening the debate on this box number 3. Madam, you have the floor.


Germany [46:11]:

Thank you, Mr. Chair. The German delegation highly appreciates the proposed text changes for Box 3 that you have circulated last night. We do realize how much of an effort it is to adequately reflect the lively discussion on Box 3 that we had on Tuesday, and we acknowledge that the text is continuing to improve, in our view. Even though the addition of the new para 2 may seem redundant by its substance, as the principles of distinction, proportionality, and precautions in attack are already included in the reference to IHL of para 1, we feel that underscoring these principles again can be very useful. Regarding paragraph 2, If we keep the new inter alia formulation, we think that we should reconsider if it is indeed necessary to keep the specifications on certain topics as under 3, 4, 5, and new 6, or discuss the way we should address them. We are reluctant on these limitations as the specifics of time, geographical, target limitations, and the advantages of machine learning capabilities are highly systems and context dependent. We proposed some text on this part during the first reading of box 3. Furthermore, in our view, paragraph 7 also captures the idea behind the formulations in new Roman numeral 6. As box 3 is at the very center of our discussions, we look forward to the discussion. Thank you.


Chair [47:47]:

Let me thank the distinguished representative of Germany for sharing her views and starting this debate. Who would like to follow up? Let me now pass the floor to the distinguished representative of Switzerland. You have the floor.


Switzerland [48:40]:

Thank you, Mr. Chair, and also thank you for your introduction of Box 3 and the changes you made. We appreciate that the text maintains what we see as credible prohibitions and restrictions and limits that contain pragmatic and implementable measures in order to preserve human control where needed, in order to ensure compliance with IHL, and to address the ethical, legal, and operational challenges posed by autonomous weapons systems. It is positive that we build on agreements and translate them into elements in view of concrete normative and legal commitments. With regard to the first two paragraphs, we support the merger and the addition of a positive obligation. That said, While the first paragraph in our review is more focused on weapons law, the second paragraph restating a positive obligation relates more particularly also to targeting law, and we believe it's important not to limit such a statement only to the rules and principles of distinction, proportionality, and precautions. Rather, It should clearly state that autonomous weapons may only be used in accordance with the rules and principles of international humanitarian law, including distinction and so on. This is important because the prohibition under paragraph 1 only concerns systems that as such cannot be used in accordance with IHL as indicated by the terms "by nature", inherently or incapable. If paragraph 2 would be limited to the fundamental principles of the conduct of hostilities, numerous others, sometimes very specific rules of IHL, would risk to be neglected. On paragraphs 3 and 4, We support that they remain basically unchanged. We also support that "developed" was added to paragraph 4. Paragraph 5 is a key element for us, and this is the key point almost of the entire text, and the basis for what follows in paragraph 6 and for other measures so far identified. While we are looking forward to more frank and fruitful discussions about context appropriate human control and judgment, and while the concept certainly needs further clarification, we are also quite convinced that for the moment no better formulation of the concept has been found. Quite the opposite, We thought this week was one of the positive elements to see how more and more delegations can support this concept. In that respect, we think it's positive that you left this untouched for now. With regard to paragraph 6 and its chapeau, we think there is overall a very positive development. While we still have some question mark about the absolute necessity to have a chapeau, a question which will certainly also depend on the eventual form and structure of the instrument. For now, it, however, gives us guidance for our deliberations for identifying measures and restrictions deemed important with regard to autonomous weapons systems. In any case, We would be of the view that the text should speak of measures that must be taken so that, notably, IHL is respected and not so that it can be respected. Respect for IHL is not an option, but an obligation. The chapeau should thus read, "to ensure that laws are used," and so on. Regarding 6C, We understand the logic behind the introduction of an additional verb to that of limit, but are not sure that the word control is the most appropriate. It creates a confusion with the notion of context-appropriate human control. A verb like define or delineate would be more appropriate. We also share the view that limiting the number of engagements that laws can undertake per attack prior to further context-appropriate human control and judgment would be a useful and important precautionary measure. We would prefer a formulation that specifies this clearly, but autonomously, as you put, could also be an option. although we might need to have a common understanding on what autonomously actually means. Like a number of other delegations, we consider that the use of autonomous weapons in concentration of civilians give rise to specific challenges for the respect of IHL. In this context, we deem the consideration contained in 6C.4 to be of particular importance. A notion that may be missing somehow is that autonomous weapons are nonetheless employed in concentration of civilians. Special care must be given to the implementation of the other measures listed under 6C. In addition, and I believe some delegation proposed a wording to limit the operation of loss to a, a defined perimeter or by defining no strike areas, uh, in particular, uh, with a view to avoiding and, in any event, minimizing incident loss. of civilian life, injury to civilians and damage to civilian objects, we believe that such an addition could also provide a useful precautionary measure. Finally, on paragraph 7, we think this is a good fix which we can fully support. Thank you very much, Mr. Chair.


Chair [56:17]:

Let me thank the distinguished representative of Switzerland for sharing the views of his delegation and for proposing some changes in the text. Who would like to follow on? I recognise the distinguished representative of Ireland. So you have the floor.


Ireland [56:36]:

Thank you, Mr. Chair, and we want to join others in thanking you for this revised box three and for your hard work as well of your team, noting the large volume of suggestions that you've had to consider and thread a needle through in presenting the latest revision of the text. And we do commend you for your overall approach. In general, we are supportive of the changes and in most cases believe that they strengthen this important section. And we'll just make a few initial remarks on this from our side. We welcome the inclusion of "develop" in paragraph 4, and we think the term "develop or use" also belongs in paragraphs 1 and 5. Paragraph 6, we view the reformulated chapeau here as fundamental. It allows us to create that bridge and provide more clarity by what we mean by context-appropriate human control and judgment in practice, as many delegations have noted throughout the week. These are the spatial and temporal limitations necessary to exercise meaningful human control over laws. And here they are spelled out to provide clear meaning and thus avoid possible differing interpretations of context appropriate human control and judgment. This is a key section as part of the two-tier approach in our view. On 6B, we welcome the inclusion of ethical, and think it also makes sense to change during to with regard to, as you have done, which does help clarify the type of human oversight needed for these unique weapon systems. On 6C, we support the addition of control or in the chapeau, although hearing the Swiss intervention just now, and also thinking it may contribute to a circular argument taking into account the chapeau, so we might just think about this further. And finally, just one small suggestion on 6C, Roman numeral one. We think it could read deactivated in a timely manner, and that's just a matter of syntax. Thank you, Mr. Chair.


Chair [58:52]:

Let me thank the distinguished representative of Ireland for his comments and suggestions with regards to the text in front of us. Next on my list is the ICSC, to be followed by the Asia Pacific Institute. Let me first pass the floor to the ICSC. You have the floor.


ICRC [59:11]:

Thank you, Chair. Good morning. We would also like to add our voice to commending you for the hard work undertaken to update and circulate this text, and also to thank you for the helpful explanation you've provided this morning about the changes. Overall, we very much support the changes made, and we see the text as moving in a very positive direction. On paragraph 2, we would support the text put forward this morning by the distinguished delegate from Switzerland, since the whole body of what is referred to as targeting law must be complied with in the use of any weapon system, including autonomous weapons. And this extends beyond only distinction, proportionality, and precautions in attack. On paragraph 4, we welcome the inclusion of the development reference in this prohibition. Applying the prohibition to development reflects the need to ensure that weapons are not developed that would likely be incompatible with IHL, and we think it provides greater clarity to industry. We echo Ireland's proposal to also include this reference in paragraphs 1 and 5 in line with our earlier interventions. On paragraph 6, we see the changes that you've made to the chapeau of this paragraph as very useful. They frame the specific measures that follow as being the way in which compliance with IHL is supported, and also the way in which context-appropriate human control and judgment is achieved over the effects of non-prohibited autonomous weapons in practice. Again, we would support Switzerland's language this morning to reflect the mandatory nature of IHL compliance here, though we would add only. So the text would read, to ensure that laws are only used in compliance with IHL. We have noted previously in this forum and elsewhere that ensuring human control and judgment in the use of force flows from key legal obligations, and here perhaps I might respond Briefly, to some of the questions you posed yesterday, the ongoing IHL obligation, for example, to cancel or suspend an attack if it becomes apparent that a target is not a military objective, this necessitates the exercise of human control. The need to anticipate and weigh the military advantage as part of a proportionality assessment, this calls for human judgment. In our view, the level of specificity of the measures that are in subparagraphs A to C serve to make it clear for those reading the text that what the group intends when referring to context-appropriate human control and judgment in the chapeau is not necessarily direct control over the weapon system at all times, but rather over the operating environment and the parameters, and consequently, over the weapon's effects. And we appreciate your drafting, which further reinforces this in subparagraph A, for example. The level of detail in these controls and limitations from subparagraphs A to C also has the potential to give actionable guidance and clarity to those using these systems. We see this as being the value of this group's work. We have consistently urged the group to move beyond restating IHL and to elaborate how IHL rules apply to the development and use of autonomous weapons. And we do think that particularly paragraphs four, five, and six of this box currently do that. We see this as very much in line with the objective of the CCW and indeed the role of high contracting parties namely the progressive development of IHL as applied to weapons. IHL is not a static body of law. It has always developed and adapted to respond to the evolving nature of wars and weapons. And considering the significant new implications brought about by the way in which laws function, which have led to over a decade of discussions now by this group, we find it appropriate that the law now must respond to these new challenges. Accordingly, the fact that something like the concept of context-appropriate human control and judgment does not appear explicitly in existing IHL, even though we argue that it is required by IHL's rules, this should not be a barrier to its inclusion in an instrument developed by this group, especially when such an element is then usefully elaborated with practical weapon-specific guidance. Chair, looking at subparagraph 6, we would suggest amending the reference there that ties the obligation to identification and/or engagement stages, and instead to refer to the selection and engagement stages, in line with our earlier comments on box 1, that these functions of selection and engagement are the critical functions for characterisation of an autonomous weapons. In any case, and at the risk of getting into another cumulative debate, we do not see a need for or in this line. Chair, whilst we reiterate these comments that the box in this, this text in this box is in a very good shape, we do see a need to again stress what we see as a key gap in the current list of prohibitions, and that is the lack of an explicit prohibition of anti-personnel autonomous weapons. In line with our earlier comments, we see a complete prohibition on anti-personnel autonomous weapons as necessary to address serious risks of IHL violations, as well as ethical concerns and risks of proliferation and divergence that are particularly acute with such systems. We also have the minor grammatical point that Ireland has already well made in paragraph 6C, Roman 1. There appears to be some words missing, so we'd support ensuring that laws can be deactivated in a timely manner, but that's a minor issue. Again, thank you, Chair. We are very appreciative of the strengthening of the text, and we commend your good work.


Chair [1:05:16]:

Let me thank the distinguished representative of the ICRC for commenting on the text and making proposals to the changes we have introduced. Next on my list is the Asia Pacific Institute to be followed by France. But let me first pass the floor to the distinguished representative of the Asia Pacific Institute. You have the floor.


Asia Pacific Institute [1:05:42]:

Thank you, Chair. First of all, many thanks to you, Chair, your team, and the friends of the Chair for all their work on this revised text in box 3. Paragraph 1 is a concise reflection of IHL in relation to inherently unlawful weapons, and addressing here weapons that are otherwise incapable of being used in compliance with IHL is appropriate for completeness. And this change also avoids the circularity of the earlier paragraph 1, which has now been deleted. Paragraph 2 is workable as it currently stands. However, as Switzerland and the ICRC have pointed out just a few moments ago, there are other rules and principles of IHL that are relevant to the use of laws, but that are not captured here explicitly. We therefore support the proposal of Switzerland in this regard. Paragraph 3 seems unproblematic from our perspective. The text mirrors, wisely we think, existing law, notably the phrasing of Articles 51 and 52 of Additional Protocol I. Chair, paragraph 4, in our view, risks conflating an existing legal obligation and the conditions or measures necessary to comply with that obligation. IHL prohibits the use of weapons the effects of which cannot be limited as required by law. This prohibition is reflected in Article 51 of Additional Protocol I. Admittedly, compliance with this rule presumes the ability to anticipate and control the effects of the weapon, but this is not the direct requirement of the rule in question. We therefore suggest deleting this paragraph and reflecting its substance elsewhere. Specifically, a sentence could be added at the end of paragraph 1, which already reflects the underlying prohibition of indiscriminate weapons. This additional sentence could read, quote, "This includes weapons the effects of which cannot be limited as required by IHL," end quote. Additionally, a new subparagraph could then be added to paragraph 6, which deals with practical compliance measures. This subparagraph could read, "Ensure that they can sufficiently anticipate and control the effects of laws." Chair, paragraph 5 presents some challenges. For one, it introduces an entirely new prohibition based on the lack of context-appropriate human control and judgment. We share the concerns already expressed by other delegations about whether this notion is clear enough and sufficiently widely understood to work as a standard for a prohibition. But this is fundamentally a policy question for representatives of high contracting parties to address. In any event, compliance with the standard of context appropriate human control and judgment seems highly context dependent. This would make it challenging to assess during the legal review whether a new laws would fall short of this standard or not. Therefore, if this paragraph is retained, it might be more productive to reformulate it as a positive obligation to ensure context appropriate human control and judgment to the extent required for compliance with IHL. Chair, turning now to the quite complex paragraph six, and I'll make here just one suggestion. Subparagraph alpha lists a number of principles on the responsible use of artificial intelligence, which have been reflected in various international and national policy instruments. and therefore reflecting them as possible elements of an instrument seems appropriate. However, it would be more helpful and transparent not to use just the labels, which can be subject to divergent interpretations, but to actually formulate these principles as separate sub-paragraphs. For example, instead of saying that laws must inter alia be reliable, a separate subparagraph could be inserted along the following lines. Quote, "Ensure that laws have explicit, well-defined uses, and the safety, security, and effectiveness of laws will be subject to testing and assurance within those defined uses across their entire life cycles." End quote. And the same could be done with predictable, traceable, and explainable. And we are happy to propose detailed language to the Chair if that would be of assistance. By unpacking these notions, the group could have a more detailed discussion about their substance. The group could then also have an exchange about as to whether they are suitable as elements of an instrument and where precisely they should be placed structurally. For example, traceability and explainability might be more relevant to box five, which deals with accountability and responsibility. And finally, in paragraph seven, the use of the phrase "cannot be modified by the system" may be read as giving some agency to the system itself. Perhaps it would be better to say, "do not change." The paragraph would therefore read, quote, ensure that laws mission parameters with regard to the target selection and engagement do not change without context appropriate human control and judgment end quote I thank you chair.


Chair [1:11:43]:

Let me thank the distinguished representative of the Asia Pacific Institute. And yes, the chair would appreciate to receive your proposals in writing. I have tried to follow it, but I'm afraid that I not got it entirely on paper. And I think it would also be helpful for others. Next on my list is France to be followed by Ukraine. But let me first pass the floor to the distinguished representative of France, who is available.


France [1:12:12]:

Thank you very much, Chair. Once again, we'd like to thank you for your efforts and the efforts of your team for presenting this new text. We can support the new drafting of paragraphs 1 and 2 that cover the same ideas that were reflected in the previous version. In paragraph 4, related to the inclusion of the term "develop", we like to express our reservations for the time being. Indeed, the contours of the prohibition with the use of "or" are not really clear and may undermine research and development in laws systems. On the contrary, we believe that it will be effectively prohibited to develop in an intentional way systems that thereafter will be used in a way that is incompatible with IHL. The opinions of other delegations on this matter will be useful. On paragraph 6, the new language that we find in the chapeau is something that we can accept, and it creates a better link with the previous paragraphs and the measures that stem from those paragraphs. In 6a, adding the words "those responsible for their employment" seems to us to be a bit restrictive. Indeed, the appreciation of effects should be carried out during the entire life cycle of the systems. And those effects should be reasonably predictable before the employment phase, but also at other moments, including during the legal reviews, in order to determine whether the system is compatible with IHL. And also at the stage of pre-deployment, in order to ensure that the commanding officers can decide on whether to use and how to employ the system, so we'd prefer to come back to the previous version that was more general. In 6, we can accept the new language, the reference made to the chain of command, reiterating the idea that the identification and engagement of the target is carried out on the basis of legal and operational considerations. and the determination of the military nature of a target is only done on the basis of legal considerations and not ethical considerations. And that's why we ask you to delete the words "ethical." On paragraph 6, we would like to once again see the inclusion of the expression as appropriate that will apply to the measures that are described later. The definition of the framework for the employment of laws and the modalities of such employment will depend on different factors and will be decided on a case-by-case basis. The functional capabilities of the system, applicable rules, and the decisions of the command. We believe that "inter alia" does not help us capture that idea, and we would like to hear what other delegations think about this. We continue to request the deletion of subparagraphs, and. If we are in favour of the evolution of IHL in order to ensure that it can deal with new challenges. At the same time, we do not believe that laws should be submitted to an exceptional legal regime through new rules that will undermine the common core of IHL. In this respect, limiting commitments to military obligations by nature, even though those are much broader or limiting the number of engagements. This could represent some useful precautions to reduce harm to civilians and could not represent general prohibitions under IHL. We remember the idea that was proposed by Germany last Tuesday on having new language to focus more on measures that can be taken by the command to facilitate the implementation of international humanitarian law. And finally, on small Roman numeral six and paragraph seven, we believe that the modification of parameters or machine learning can be part of autonomy. Mission parameters should, though, not be modified from the framework defined by the chain of command. So these paragraphs should be made more precise to reflect that idea, for example, through some language such as.

Ensure that laws cannot modify their initial programming or mission parameters without any human validation.

And we can send you that proposal in writing, and thank you very much.


Chair [1:18:40]:

Yes, indeed, that would be very helpful, and I would like to thank the distinguished representative of France for sharing her views and making comments and suggestions. Next on my list is the Distinguished Representative of Ukraine, to be followed by Austria and Pakistan. But first, the Distinguished Representative of Ukraine. You have the floor.


Ukraine [1:19:01]:

Thank you, Mr. Chairman. Thank you for your version of this box and your big work on updating the text. We agree with paragraph from one to three. It is good for us and for our opinion. Also, we confirm that including new word "develop" to paragraph four is good. But in my opinion, I said about this before, that this paragraph should be included similar to point of paragraph 6A, adequately prediction, because I said, USA representative, we cannot forecast behavior bullet after shoot. If we have countermeasure or opposite of enemy. Please take into account this conclusion. Next, regarding paragraph five and six and seven, in my opinion, term human control also included judgment. In this regards, we should be delete word judgments from box three and keep only human control. And the last point, Ukraine delegation support position of France regarding paragraph 6C concerning deleting position Romanian 345 and regarding new addition common text for paragraph 7 and C 6C6. Thank you.


Chair [1:21:18]:

Let me thank the distinguished representative of Ukraine for sharing his views and making comments. Next on my list is Austria, to be followed by Pakistan. Let me first pass the floor to the distinguished representative of Austria. You have the floor.


Austria [1:21:33]:

Thank you very much. And also from our side, our heartfelt thanks for the hard work that went into the text last night, and to you, and of course, to all of your team. Going into it, we've seen a lot of very useful changes to the text that make it better, that go into the thoughtful discussion that we have been having yesterday to some of the concerns delegations have brought up and how to address them. And it always meets the middle ground in most cases. And we are just trying to fixing some of the things that we see as still being necessary and further improving the text. Going into para 1, we would also support to include the development use instead of the employ. It has also been suggested to other paras, but in this one, I think it's the most important one, because what we are doing here is we are reaffirming a norm that exists in IHL, and we are reaffirming it in relation to loss. And as we all know, we are going with this technology now to a direction where we have different stakeholders, other entities that are also involved in how these weapons are developed. And this just gives some additional guidance to this level. Developers will have to know that this also applies to them and that this IHL rule is relevant in their work and not just when the system is used. We hear the views that this could be difficult because they might not know that their system could then violate this or that IHL rule. The issue is they should know, they should be aware that this IHL rule exists, and that is why this is relevant. If they are never informed that there is this problem, we are not doing our job properly. The second point is that When it comes to this idea that you don't know at the development stage what it will be doing at the end, this is a problem that we know, or it's not necessarily a problem that we know from different conventions in the IHL sphere. In the BWC, we have a very specific clause for peaceful purposes, and this includes the idea of intention. So if you are intending to develop a system to cause superfluous injury or unnecessary suffering or which are inherently indiscriminate, this is obviously prohibited. You can claim that this was not the intention, but we can have this element that this is something that has to be prohibited already at the development stage here. And this is maybe something to discuss so that we are all on the same page of what we are talking about. If you don't have the intention or not even the knowledge that what you are doing is violating an IHL principle, then of course you're in a different stage and you're probably not affected by this para, just to make this argument. On para 2, we support the Swiss comments, also supported by others. When it comes to 4, here you also have inserted to develop the understanding that we have of the insertion here that it relates to possible unpredictable weapons, so there is a certain usefulness as well, but you also understand the comments, and maybe this is something that we can further discuss, but we are very happy as it is now. The question is, as militaries or as other actors, do you actually want to develop a system where the effects in attack cannot be anticipated and controlled as required by IHL? It's also very clear that it will be prohibited anyway. And again, it's the same argument as before about the intention or non-intention in the development stage. Five, for us, is a very important para as it reflects the tutti approach and is kind of the bridge from the first four to the next stage. A much more important bridge is, of course, para 6, and we very much appreciate the changes you have made to the chapeau. Our understanding is that this gives flesh to human control, something that has been criticized in the course of this week and also this morning, that it's not clear what it means by saying how human control and meaningful or context appropriate or context and meaningful, context appropriate and meaningful human control and touchment, how it can be achieved. There's also the insertion compliance with IHL, so there's also a strong link of all the measures that follow that this is the chapeau indicates that this is for the purpose of compliance with IHL, and I think this also alleviates many of the concerns that we heard, and it makes just much more clear what states have to do. The third point, and we have made this Repeatedly is that those measures that follow are also for the purpose of allowing operators to, um, actually have human control. If we have this element somewhere else, it allows them to be able to control a system that goes above their mental capacities to some extent and the cognitive, the simple basic human cognitive capabilities. And by bringing down the engagements, the types of targets, duration and so on, humans will be actually able to follow. Otherwise, we will have uncontrolled weapons and as you all know, those are perceived forbidden by international and humanitarian law. Another, we are happy with many of the changes, or at least we think they are useful to address some of the concerns raised by others. One notion that we were interested to have also under six is the one about being able to make informed judgments. Our proposal for another large letter para after B was to ensure that operators are able to make informed judgments about the anticipated outcomes of the use of force in accordance with legal obligations, including through measures to mitigate automation bias. The element that we would bring in here is the training element or the requirement or the measure. It's not a requirement as it is set out here. The measure to ensure that operators have the capacity to actually understand the anticipated outcomes and make informed judgment. And it would actually bring in the judgment work back into the equation, which some might appreciate. We can go along with ethical instead of moral. The with regard to change is also a very useful one. It creates-- it solves the problem that we might not even have seen in the previous round. see inter alia is a very important addition as it makes clear that the following list is open-ended and that there are also other measures states could take, which we here don't necessarily spell out. On the question for deletion of several of the Roman numbered paras, we would like to keep them. We think all of them have their own purpose and own sense, especially limiting the number of engagements. As some have pointed out, this is probably an operational requirement. And one, as I have said, given the big cognitive difference between AI and humans, you could have thousands or millions of of engagement, you're thinking here about algorithmic operations in a few seconds, not necessarily an engagement. We all know that this takes a little bit longer, that you will not be able to follow. So limiting engagements is just, in our view, a precaution that help states to be able to exercise human control. And just to point out that, inter alia, is an important addition, that this is something that states can do. And maybe the last point on Roman 6 and 7. We appreciate Roman's six. I mean, it adds something that has been discussed more or less on the sidelines in the GGE laws, but which is an important question and the one if machine learning should be live or real time, if machine learning or changing the system by itself and its operation, its functions, or even the algorithms should happen in an operational context. And I think many will agree that it should not. Therefore, we see this as something very, very important. I mean, in the end, we could even say that we could even make in a prohibition and say that this should not happen, that machine learning should only happen when the system is in the garage to say so and to take in the operational data that it has gathered when it is at home and not while it is in a in an operational context. Sorry for using the simple words for a technical issue. We also think that the issue of real-time machine learning is different from mission parameters. So both of Roman 6 and 7 both have their own justification. And here, the French proposal and the French changes are pointing to that, because what we want is not that-- it not relates, seven does not relate to machine learning, but to mission parameters. And mission parameters, when it comes to the implementation of IHL, don't only relate to regard to the target selection and engagement functions, but it could also relate to the operation area, the geographical scope. An autonomous weapon system should not change the mission area just because it thinks, okay, over there, there might be much more interesting targets. So the geographical dimension, which I mean, you could argue that this has to do with target selection and engagement functions, but we would link it much stronger to the mission parameters. It's the most simple mission parameters. This is operation area, don't go out of it, and the system should not be able to change it. So if we limit this down to the target selection and engagement functions, it could create some problems. And mission parameters are a much broader concept, and the French proposal solves this. And at the same time, we could keep the machine learning under Roman VI. And this was my last point. Thank you.


Chair [1:32:36]:

Let me thank the distinguished representative of Austria for sharing his views, comments, and making proposals. Next on my list is Pakistan, to be followed by Belarus, India, and Singapore. But let me first pass the floor to the distinguished representative of Pakistan. So you have the floor.


Pakistan [1:32:52]:

Thank you, Chair. We join others in commending you and your team for this revised text. We really appreciate your approach that you have taken in this box, which mirrors the approach in the previous boxes to work on those select narrow areas which could help build further convergences at this stage. Regarding paragraph 2, the new paragraph 2, we think that the proposal made by the Swiss delegation and supported by ICRC makes a lot of sense. And we are also not sure regarding the placement of this paragraph here. We understand that the first part of this box is mostly focused on prohibitions. So perhaps the one that focuses on the use in accordance with the targeting law could come in the later half of this box towards the end. Then on the issue of develop or use, we welcome that the word develop has been introduced in paragraph four, but just as highlighted by colleagues from Ireland and Austria, we believe that it should be there in paragraph one as well as in paragraph five. And this is again to address the fundamental question that we have been raising and other colleagues have been raising that how to address those weapon systems that by their design or development parameters cannot be used in compliance with IHL. And if we limit our focus only on employment, then we deliberately ignore the very processes that could determine whether a weapon can ever comply with IHL. And this is also linked with the conversations that we have had this morning and yesterday in the context of appropriate human control. And we have been focusing on not only during the engagement cycle, but also before the engagement cycle, how certain safeguards could be inbuilt in those weapon systems. Therefore, we strongly support the phrase of developer use in all these places. Then we really appreciate that you have retained paragraph five on context appropriate human control and judgment. We believe that together with six that offers greater clarity. And then when we go to six, we think the is much improved. And it takes into account the comments made by several delegations of how to further clarify what follows in paragraph six and linking it firmly with the compliance with IHL and, of course, with context appropriate human control and judgment, how that could be spelled out in this place. We can support the proposal by the Swiss delegate here as well regarding changing can be used to, are used to make it more obligatory in terms of IHL. We really found the explanation given by the ICRC about the subparagraphs and its relation with the very useful. And we strongly believe that the retention of all these subparagraphs is important to spell out those parameters in terms of operations of laws. In sub-paragraph a, we note and we appreciate that you have taken on board the conversations that have been taking place. And I think it was proposed by the delegation of the UK previously that perhaps these principles, predictability, reliability, traceability, could make more sense if they could be linked with those who are responsible for the employment of this. We do see the point made by the French delegate that that makes it more restrictive, and we would also prefer that it would be more broad. But at the same point, of course, this is the place that we could start with to build convergence on the issues of predictability, reliability and traceability. And here we do see the point, of course, where the delegate from the Asia Pacific Institute helped explain in terms of traceability, but we believe that traceability is not just about well-defined parameters or the listing out the explicit uses of those weapon systems. It is about how the behavior should remain stable and consistent with to the expectations under expected battlefield conditions. So it's much broader and therefore we believe that these principles should be listed as as they are listed right now. In six sub paragraph B, we would support the retention of word ethical together with legal. We believe that the group has agreed in the past in 2023 report as well and previously that ethical perspective should continue to guide the work of this group and we are just fulfilling those conclusion and recommendations that the group has made in the past. In six paragraph C and it's Further, the Roman paragraphs, we believe that now with the SHAPE-O control or limit, it gives more flexibility. We are talking about the control over the types of targets, duration, geographical scope, and all the Roman numerals, paragraphs, which are listed are very important. And I would just only speak perhaps on Roman III and Roman VI briefly. On Roman III, Limiting the number of engagements, I think it's an important concept and that some colleagues have already spoken about. If we look at the existing weapon systems, we believe that control, that limit is already there when we employ those weapon systems to comply with the IHL obligations. When we are Complying with the targeting law, for instance, the distinction, proportionality, or precautions in attack, we already have certain limits and controls in place, including on the number of engagements. And given the risks and concerns that have been expressed in case of loss previously, how it could be difficult for humans to follow certain engagements if the warfare is evolving at a battle speed in the future, for instance. Therefore, we believe a reference to consideration of number of engagements and controlling those number of engagements is particularly important, especially if linked with compliance with IHL. We are very happy to see real-time machine learning here as well. And I think we can subscribe to the comments made by delegations of France and Austria on how we could further modify and retain this notion in the first place. I'll end there, Mr. Chair. Thank you.


Chair [1:40:00]:

Let me thank the distinguished representative of Pakistan for reacting to the text proposed by the Chair and sharing comments. Next on my list is Belarus, to be followed by India, Singapore, and the US. But let me first pass the floor to the distinguished representative of Belarus.


Belarus [1:40:18]:

Thank you very much, Chair. And I would like to thank you for the text that you prepared. The text reflects our discussions. I would like to try to make some proposals, perhaps. This will be some food for thought and help us resolve some of the problems. Try to give a definition to context-appropriate human control, because this is a key notion here in this box. and perhaps clear identification of what we mean will help us. I'll now switch to English.

The definition of appropriate human control of laws means the ability and the relevant process, first, to program a lethal weapons system according to certain rules of engagement, and second, to guarantee.

That the laws comply to these rules of engagement, including the period of autonomous learning of the laws and subsequent.

Autonomous identification, selection.

And engagement of target by the laws.

And third, to judge the application of the laws constantly. And fourth and the last, to modify the operation of the laws and the employed rules of engagement at any time. That's all, I hope.

And a small addition, maybe, maybe suggestion to sub-paragraph Roman 4 in sub-paragraph C of paragraph 6, avoiding concentration of civilians or.

Civilian objects. Now.

We believe that in certain situations, laws can be useful to control situations related to concentrations of civilians in order to, in some way, protect them and destroy specific targets that threaten civilians. This is why we can propose the following language:.

Avoiding that the application of laws aggravates the situation with regard to concentrations of civilians or civilians objects.

This means It's not that in principle that these systems should not be used against civilians, but they should not exacerbate the situation of civilians. Thank you.


Chair [1:44:05]:

Let me thank the distinguished representative of Belarus for commenting on the text and making proposals, and the Chair would appreciate to receive the proposals in writing. Thank you in advance for that. Next on my list is India, to be followed by Singapore, US, Australia, and then we have also Spain, Canada, and the Republic of Korea. But let me first pass the floor to the distinguished representative of India. You have the floor.


India [1:44:30]:

Thank you, Mr. Chairperson, for giving me the floor. And I would like to begin by commending your broad approach, your consultative approach and your sheer patience in listening to the various views that delegations have shared on various sections of the rolling text. My comments on this section are going to be preliminary and I will go in a one by one manner addressing the various provisions that are contained in the rolling text including the edits that you have suggested. From paragraph 1 through 4, we believe that this is a placeholder for reiterating some of the prohibitions that are already contained in the international humanitarian law, including those that are related to targeting law and those that are derived from the inherent nature of weapons as contained in Article 35 of the Additional Protocol I. we would urge that there not be any duplication of what we are trying to convey. And I would also like to stress that some states which are discussing lethal autonomous weapon systems in the context of CCW are not signatories of Additional Protocol 1, but they are still open to using the language as is contained in the various sections of the Additional Protocol I. And therefore, we would suggest that we not be repetitive in producing those prohibitions or regulations that come from IHL in various different sections of this section. We are also open to using them directly without really exposing them to a reformulation exercise where we might end up restricting or compromising the scope of the language which is contained in a legal text such as the International Monetary Law. Second, on the paragraph four where reference is made to prohibition in all circumstances on development and use of weapon systems. I wanted to come in on here and clarify that we do not dispute the fact that the international humanitarian law does apply to the development and use of lethal autonomous weapon systems. In fact, the 2018 report and the guiding principles have affirmed that IHL also applies to the potential development of these weapon systems, which also perhaps might include design. Where we come from is that when you look at the prohibitions that are contained in the international humanitarian law, both in the targeting law and what some might call as weapons law, these prohibitions restrict themselves to the use of these weapons. And that is something that is not just restricted in the international humanitarian law, the CCW protocols, and the CCW framework convention. Also, when you look at the prohibitions, these apply to the use of these weapon systems. Our view is that if we were to broaden this to development, it is possible that this will have a chilling out effect on many of these technologies, emerging technologies which are used for developmental peaceful purposes in alleviating poverty, in enhancing socioeconomic development for many of our countries. And these are the same technologies which can also be, for instance, used for non-peaceful purposes. And if we were to enhance the scope from the use which which I think is clear from the text that we have, the international humanitarian law and CCW that it is restricted to the use. If we were to expand it to the development, it might also end up having deleterious or negative impact on the access to these technologies to developing countries. So when we speak, we speak from that perspective. Also, When you look at the paragraph four, there are terms which have been used pertaining to anticipated effects, anticipated and controlled. We are willing to give it an open consideration, but these are not concepts which have been agreed thus far in the context of the group of the governmental experts. Of course, in 2019, we've agreed to the concept of human judgment. to ensure that this judgment is translated into action, we also agreed to what is control in 2023. And we noted that both human judgment and control is necessary to ensure that lethal autonomous weapon systems are able to comply with the provisions and the principles of the international humanitarian law. We want to say that this control could be exercised in multiple ways, as was pointed out by several delegations yesterday. Human control is one way of exercising that control. There are other ways, such as those through software, through automation, through inherent limitations of capabilities of these systems. And that the element which is necessary, which is of the human element, is sufficed by the or is sufficiently directed through the human judgment which would ensure that factors such as the characteristics of the weapon systems, the operational environment in which these weapons are used, the context specific factors are taken into account through the human judgment and through the concept of control we are able to translate this human judgment into into practical action. And for that, we believe that in case of paragraph 5, where we have used control, human control and judgment, we have packaged that as a form of a prohibition. We would strongly urge that this be used as a form of regulation instead of as a prohibition. There is a manner in which the group had agreed to this in 2019 and 2023. We do not automatically see them to be presumed as conditions on the basis of which a prohibition could be packaged out of them. On paragraph 6A, we see that there is a reference which is made to a number of new concepts and terminologies. We ourselves, uh, do believe that some of them pertaining to predictability and reliability are, uh, are, uh, as some pointed out, uh, if interpreted in, uh, you know, in the, uh, in the framework of the… IHL, are some important qualities and objectives and attributes which are necessary as far as the lethal autonomous weapon systems are concerned. Although we do believe that the interpretation of these words would actually come from what states believe is-- they're going to be interpreted in the national framework. But we also would like to emphasize the fact that terms such as predictability and reliability, at the end of the day, are non-objective terms. countries will have different interpretations that they make on these terms. They do not have objective criteria. We can still go ahead and have reference to these terms such as predictability and reliability, but those pertaining to traceability and explainability, which it is our view that given the evolution of the technology right now, some of these terms we would have some objections to being referred to in this section. We are open to recognizing the imperative of traceability and explainability and referring to them in the section on risk mitigation measures. But at this point of time, we do not believe that sufficient accretion of understanding has been achieved where we can crystallize them as a part of the regulation measures. Likewise, on the point 6B, we have agreed in the preamble of the guiding principles that ethical perspectives do guide our work. And it is therefore our view that if we are to refer to the ethical perspectives in the context of our work, they should rather be, they should, like the way they were agreed in 2018, be mentioned in the section two of our rolling text, where we can affirm that relevant ethical guidance should guide the use of these weapon systems. In addition, we believe that there's been some useful work which has been done by the group in the past pertaining to factors, concepts such as human machine interaction. And we, in the guiding principles, see we have agreed that human machine factors, human machine interaction can be implemented in various stages of life cycle. The quality and the extent of these human machine interaction depends on the operational context. It also depends on the characteristics and the capabilities of systems. So we do believe that some of these understandings which the group agreed to in the past can also be provided in the rolling text because they elaborate the context in how the understandings looked at the various factors of human machine interaction, which is that there's no one-size-fits-all approach. We do understand the imperatives of the imperative of human judgment as was agreed in the group. We also agree on the one size fits approach of control. But at the end of the day, how human machine interaction has to take place and what factors it will be dependent on, given the variance of operational context, given the variance of the quality and extent of human machine interaction, and also the characteristics and capabilities of the system have to be identified through human judgment and the exercise of control. Likewise, the 2019 report referred to in its para 23A that States must take measures to minimize the risk of incidental loss of life, which are to be exercised during the life cycle of weapon system. It also refers to reducing the risk of unintended engagement, civilian loss of control of system, risk of proliferation, and risk of acquisition by terrorist groups. I think that these are some important-- this is, again, an important understanding which which the group has agreed to and which we should not have objections if they find a place in the section on regulations. And I think much of our discussion which have taken place have started from the slate that lethal autonomous weapon systems would negatively impact international humanitarian law. But I think it's also important to look at the other side of the picture that there are also views that the autonomous weapon systems in some cases might enhance compliance with the international humanitarian law. So it's important that we have a well-balanced discussion where all perspectives are considered in a comprehensive and a balanced manner. I'll stop at this, Mr. Chairperson. Thank you.


Chair [1:55:51]:

Let me thank the distinguished representative of India for sharing the views of his delegation and for extensively commenting on the proposals we have made. Next on my list is Singapore, to be followed by the US. But let me first pass the floor to the distinguished representative of Singapore. You have the floor.


Singapore [1:56:11]:

Thank you, Mr. Chair. My delegation wishes to thank the Chair and your team for your efforts to address the many comments made by delegations in Box 3. We have two reactions to these changes. First, we noted that the word "deploy" in paragraph 4 has been changed to the word "develop". The current formulation seems to require States to be certain about whether the effects of the laws in attack can be anticipated or controlled when the system has not even been developed yet. Please allow my delegation to illustrate why this formulation is not consistent with current real-world development processes. Consider the concrete example of vaccine development. If a vaccine in development showed indications of unanticipated side effects, the scientists would strive to understand the underlying root cause and conduct iterative reviews of the developmental process or technique to eliminate these unanticipated side effects. The scientists will not stop developing the vaccine at the first sign of unanticipated side effects. The vaccine development process is similar to that in engineering projects, whether military or civilian. There is an iterative process of technical reviews, which is meant to improve safety, performance, legal compliance, timeliness, and/or cost-effectiveness. If the reviews show that a system is not performing up to safety requirements, the development team does not simply terminate the project immediately. Instead, the team requests for resources to focus on improving its safety. This process of reviews takes place until the system is ready for use, or terminated if it cannot be used in the way it is designed to, such as the ability to meet safety requirements. Returning to the text at hand, It appears that the purpose of paragraph 4's current phrasing is to prohibit the development and use of laws that are intended to be used in a manner that is non-compliant with IHL. To lend clarity to this paragraph, we therefore suggest a small but meaningful addition, given our earlier explanation on the development process. The paragraph would therefore read: It is prohibited in all circumstances to develop or use laws which are designed so that their effects in attack cannot be anticipated and controlled as required by IHL in the circumstances of their use. So note the addition of "designed so that." Second, my delegation also notes the addition of a new paragraph, 6, and some edits to paragraph 7. We note that paragraph 60, Roman numeral 6, prescribes limits on real-time machine learning with regard to target selection and engagement functions. At the same time, what seems to be similar prescriptions on target selection and engagement functions are reproduced in paragraph 7, which indicates that laws, mission parameters with regard to target selection and engagement functions cannot be modified without context-appropriate human control and judgment. So we would be grateful for the views of other delegations as well as the Chair regarding the substantive differences between restrictions in paragraph 6C, Roman numeral 6, and paragraph 7. I thank you, Chair.


Chair [1:59:32]:

Let me thank the distinguished representative of Singapore for sharing his comments and making suggestions to the text circulated. Next on my list is the US, to be followed by Australia and Spain. But let me first pass the floor to the US. You have the floor, sir.


United States of America [2:00:05]:

Mr. Chair, of course, we want to thank you and your team for your efforts on this revised text. We've made a number of proposals and comments on the prior draft of this section and expressed them fairly. fundamental concerns. I think we've appreciated how you've recognized there's more work to do on some of the key concepts here. I would just recall these comments and proposals in a very general way and not repeat them specifically again, but from our delegation's perspective, you know, many of them do seem quite pertinent. And we welcome the opportunity to discuss some of these further with you and with other delegations. But did want to offer some new additional specific suggestions and comments on the revisions. So first, appreciate your edits with regard to paragraph one and the deletion and then the amendment to paragraph one. Also, appreciate the new paragraph 2. Reacting a bit off the cuff to the comments from our Swiss colleagues, I think the general issue we would spot with regard to their proposal is that states are parties to different legal instruments. So what is understood by IHL, of course, varies from state to state. Sometimes this is captured by adding the word applicable to reflect the notion that states of different treaty obligations, to the extent we're also going to refer more generally to IHL rules, to us it makes sense to really focus on what colleagues have been talking about, targeting law, the rules governing attacks, rather than to reference all of IHL and rules that would not be relevant. We also did want to comment on the change from principles and requirements, which has been used in previous GGE reports and has found consensus, and the proposal with regard to rules and principles. As my delegation has indicated, we're always happy to try to improve previously agreed language if there's consensus to do so. From a substantive perspective, my delegation does not regard precautions in attack as a principle. Rather, we see this as a rule emanating from the principle of proportionality. So the specific edits I think we might suggest to address this issue would be first to delete "rules and" before "principles of distinction and proportionality." So we would say, "Laws may only be used in accordance with the principles of distinction and proportionality." And then to say, "and the rule obliging a party to a conflict to take feasible precautions in planning and conducting an attack. So that's a proposal to be more specific and reflect our understanding of this particular IHL rule. The more general comment we would like to make with regard to this paragraph and the next several paragraphs is really to endorse, I think, what our ICRC colleagues were expressing as a desire in this section to clarify and strengthen our understanding of applicable IHL. I think our general concern with some of the current language here is that if we refer very generally to IHL, As a whole, this won't give enough specific guidance to states, to parties to a conflict, to individuals who are tasked with implementing this instrument as to how and what specifically they're supposed to do. So I would recall our general desire to include more specific IHL rules and provisions. With regard to paragraph four and the word develop, I think we wanted to engage in the broader thematic discussion that colleagues were having about whether it's appropriate to add and development in here and many other places in the document. Certainly we've agree with the importance of addressing development and development activities in this document. But we've been listening to other delegations, and the only IHL rule that we have heard applicable to development that's been mentioned in these discussions is Article 36 of the First Additional Protocol to the 1949 Geneva Conventions. And that article provides in the study development, acquisition, or adoption of a new weapon, means, or method of warfare, a high contracting party is under an obligation to determine whether its employment would, in some or all circumstances, be prohibited by this protocol or by any other rule of international law applicable to the high contracting party. So we see this as a rule only applicable to high contracting parties of the 1949 Geneva Conventions. We also see this as an obligation to make a determination. So for us, it doesn't say prohibit. And so that's, I think, a fundamental issue we would note with regard to paragraph four. Paragraph four is framed in terms of prohibition, but Article 36 relates to a determination. And nor Do we think this should be made into a prohibition as the distinguished representative from Singapore just explained? As a practical matter, it's difficult to know whether it's prohibited while we're developing something. With regard to the next several paragraphs, I think the specific suggestions I would offer with regard to paragraph 6 would be to add "help" before "ensure." to delete with context appropriate human control and judgment and to add mitigate the risk of unintended engagements. With regard to the sub paragraphs, I'll just make a few comments on A to us. It seems less helpful. the revisions here in our view kind of take this in the wrong direction. That said, we were very intrigued by the suggestion from the -- our colleague from the Asia Pacific Institute about breaking these terms out in separate paragraphs and defining them and having a more focused discussion to see if we could reach consensus on them. So I think we would be willing to support and engage in that kind of discussion. With regard to paragraph B, we would suggest replacing "ensure that laws are operated with" "operate laws." And then I think we are having difficulty understanding the second sentence as relating to the first sentence. For my delegation, the legal and ethical considerations relate to the rules in IHL governing attacks and not to the concept of responsible chain of command and control. With regard to paragraph C, sub-roman numeral 4, the comment we wanted to make was similar to, I think, a comment that was made by our colleague from Belarus Autonomous weapons systems can be deployed around civilians, so they're deliberately deployed next to the concentration of civilians and civilian objects in order to protect those civilians from terrorist attacks or other harmful threats. So I think our proposals on this prior draft would help address this concern, but we think it would be important not to prohibit using autonomous weapons systems to protect civilians. And then with regard to sub-Roman 6, and then paragraph 7, I think we would repeat our concerns with regard to stigmatizing technology and potentially undermining efforts to enhance the protection of civilians in armed conflict. So just would like to register those concerns again. I thank you, Mr. Chair.


Chair [2:10:07]:

Let me thank the distinguished representative of the US for providing us with comments and suggestions with regard to the text, and we would highly appreciate it to receive your comments also in writing, if they not refer to earlier comments already made. With regard to my list, I have Australia to be followed by Spain, Canada, Republic of Korea, but let me first pass the floor to the distinguished representative of Australia. Madam, you have the floor.


Australia [2:10:36]:

Thank you, Chair, and with your indulgence, Australia will make at least two interventions over the course of this discussion, a first which is going to focus on these ideas of development and use, and we've heard a few thoughts on them this morning, and then later this morning or this afternoon, I suspect by now, we will talk to the broader questions of the redrafting of this box, for which many thanks to you and your team for your work on this, and in particular, can I just say many thanks to you all for your engagements today and yesterday. That discussion yesterday afternoon was really helpful.


Chair [2:11:14]:

I do apologise, but if you would like to have discussions, could you please do it outside the room and not at the back of the room, because it's really disturbing the good flow of the discussions here and it makes life for the interpreters very difficult. Please, you can continue.


Australia [2:11:30]:

I mean, I was perhaps too long in giving everyone a compliment anyway. But just to say we found it very helpful, so thank you all. So on this question of development and use, Chair, you know, noting the new text inserts develop at paragraph 4 of box 3, we wish now, in a bid to be helpful, to be clear about our position on development and use in the text. On Tuesday, we stated that Australia is open to the inclusion of the phrase "development and use" on a case-by-case basis, Our position was shared by a range of delegations on the grounds that a general find and replace approach would not be constructive and would not find consensus. The CCW, as has been pointed out this morning, aims to protect civilians against the effect of hostilities and protect combatants from superfluous injury and unnecessary suffering. It does this through prohibiting or restricting, through its protocols, the use of specific types of weapons that may either cause unnecessary suffering to combatants or have indiscriminate effects. To be faithful to this And echoing India, US, and others' cogent points this morning, Australia prefers for the text to prohibit or restrict the use of weapons in armed conflict and not prohibit development per se. Chair, this is for practical reasons. It is the process of development, including through testing and evaluation, that will inform whether the weapon can operate consistently with IHL. An important part of any weapons development is the national legal review process, which will identify legal risk. Legal reviews are an express IHL requirement. The legal review process will prevent or restrict the development of weapons which do not comply with IHL and other international law obligations. Australia and other delegations have demonstrated flexibility by being open to development and use on a case-by-case basis. Turning now to the paras, We don't agree with the proposal to insert development into para one for the reasons that we have just outlined. On paragraph four, replacing deploy with develop is not a replacement in kind. It instead expands a prohibition in a way which we do not find acceptable. Many delegations do not support para five as it stands, and to insert develop would likewise not be supported. Again, neither the CCW framework nor IHL prohibits the development of a weapon. In good news, box 4 here is to us the most relevant location for text relating to development, because it purports to regulate the stages of an autonomous weapons system life cycle prior to potential employment, as stated in the chapeau. This is by far the most appropriate location to discuss specific measures on development and will be implementable for states as regulatory measures on developers and development. And as a final note, just for completeness, we think for accuracy and consistency, deployment, employment and operated could be changed to use when they're mentioned throughout this text. As mentioned earlier, use is the term that's used in the CCW, and while employer is also used in the CCW, It is interesting to see four terms used interchangeably in this text, and so to avoid confusion, we would encourage the use of use. And finally, we listened with interest to the interventions of the Asia Pacific Institute and Singapore today, and we will consider them closely. Thank you.


Chair [2:15:16]:

Thank you. And indeed, also, many thanks for explaining your developed position with regard to developing and use. Now on my list, I have Spain to be followed by Canada, the Republic of Korea, Denmark, Israel, Russian Federation, Italy, and Brazil. I'm afraid that we won't make that before lunch, but perhaps, I don't know. Let's start with the distinguished representative of Spain. You have the floor.


Spain [2:15:53]:

Distinguished Chairperson, we are very grateful for the work that has been carried out by you and your team in presenting these new proposals to move us forwards with the compliance of our mandate. Allow me to convey our positions paragraph by paragraph. The division of former paragraph 1 into two separate paragraphs can be supported by my delegation in favour of consensus, although we understand that they retain the same meaning and interpretation. In terms of current paragraph 4, we can support the change to the verb to make it develop, given that that is better understood by a wide range of delegations and should be understood to include the entire life cycle of the weapon. In paragraph 6, which is essential, we think the new chapeau is very welcome, given that it provides a scope for the implementation and application of the rest of the subparagraphs. Allow me to reiterate my support to context appropriate and human control and judgment. In paragraph 6, section A, we don't have any reservations concerning the new drafting. On B, we support the inclusion in English of--.

Chain of command and control.

We also support the removal of "moral" and its substitution with the term "ethical", which is much more appropriate and it reduces the number of possible interpretations. On subparagraph, we retain what we said in our previous statement on this box. and we support, as has been said by the delegation, such as France, the removal of the third and fifth subparagraph. We would also prefer to take out the fourth, although we retain the proposal we made in our previous statements on this. In terms of new subparagraph 6, we would prefer to remove it, given that we understand that if there is control and judgment throughout the life cycle, we shouldn't understand this to be a risk, as referred to here. Now, under 7, we support the proposal of the new drafting with the same argumentation that I have just conveyed to request the elimination of subparagraph 6 of the former paragraph. Thank you.


Chair [2:18:25]:

Let me thank the distinguished representative of Spain for sharing the comments of her delegation and making proposals with regard to the text. Next on my list is Canada, to be followed by the Republic of Korea. But let me first pass the floor to the distinguished representative of Canada. Sir, you have the floor.


Canada [2:18:45]:

Thank you, Mr. Chair. And thanks again for all the work that's being done on our text so far. My delegation would like to offer a few comments on this section. Firstly, in paragraph 3, we have the same issue that we had previously highlighted, as it does not account for civilians who are directly participating in hostilities. We support the suggestion that has been made to add "individual civilians protected as such". On the discussion we're having regarding development, similar to the interventions made by Australia and Singapore, my delegation believes that development and use should not be placed together in all circumstances and needs to be evaluated on a case-by-case basis within its context. For example, we would not support the suggestions being made to add development in paragraph 1 and 5. In paragraph 6 in the chapeau, we agree with the French delegation that measures identified within this paragraph should be addressed as appropriate to the context and environment in which laws is proposed to be employed. On 6A, we remain of the view that adequately is vague, and we believe this needs to be linked so that the criteria of predictability and traceability, et cetera, ensures that the laws is capable of complying with IHL. On 6B, we believe that we could add an accountable in front of responsible chain of command and control to emphasize the extent of responsibility amounts to individual and collective accountability in the event of inappropriate use. In this paragraph, we also welcome the inclusion of ethical. In 6C, Roman numeral 1, timely remains a bit imprecise in our view. If the purpose here is deactivation, we could describe under which circumstances this would help determine what is meant by timely. Roman numeral five, when we say military objective by nature, we could expand this a little bit to include nature, location, and purpose. Finally, my delegation believes that paragraph seven is another restriction and could be included into paragraph six as Roman numeral seven. And Mr. Chair, we'll submit these suggestions in writing as well. Thank you.


Chair [2:21:15]:

Let me thank the distinguished representative of Canada for providing comments and also for writing them in writing. Next on my list is the distinguished representative of Korea. You have the floor.


Republic of Korea [2:21:27]:

Thank you, Mr. Chair, and happy Friday to all. My delegation wishes to express our deepest appreciation to you and your team for providing an updated box 3. I have five points. First, we support the revision of the first paragraph and the addition of the second paragraph. However, agreeing with Singapore, US, Australia, Canada, we do not support adding the notion of development here in the first paragraph, with other paragraphs just to inform the developers that weapons systems need to be in compliance with IHL. Such assurances could be arranged in other parts of the document, as already addressed in the box 4, or through other measures. Here, affirming the most fundamental IHL rule that is regulating the use of weapon systems is the most straightforward way and it should be our goal. Second, we are compelled to reiterate our concerns regarding paragraph 5, which, as you noted, remains intentionally unchanged. As our distinguished colleague from Australia mentioned, many delegations do not support this paragraph, and this paragraph must either be deleted or significantly revised to be acceptable. Third, regarding paragraph 6, we welcome the revision of its chapeau. However, we strongly recommend deleting the second part of the sentence and with context-appropriate human constraint judgment for two reasons. First, our primary focus here is Again, compliance with IHL and the reference to human control and judgment is at best an intervening consideration, or to some, not a requirement at all for IHL compliance. Second, this term lacks consensus in the room, as you see, and its repeated inclusion throughout the document compels many delegations to reiterate their objections or concerns, only prolonging and distracting our work. Fourth, we support the revision of 6B, replacing it with under responsible chain of command and control. We believe the suggestion made by India is useful regarding the second sentence of this paragraph, that is moving the ethical discussion to box 2. Lastly, regarding 6C, While we welcome the inclusion of "inter alia", we still think the addition of "when necessary" or "as appropriate" is needed, as suggested by many delegations during our initial discussion and today. Moreover, we find subparagraphs 3, 4, 5 and new 6 to be superfluous or unnecessary, as the main substance of this paragraph is already addressed in the chapeau of 6C. we believe that we should not introduce overly restrictive measures that do not reflect operational realities, possible future development based on abstract ideas. Furthermore, this may unduly hamper positive technological development in autonomy. I thank you, Chair.


Denmark [2:24:55]:

First, we join others in commending you for your work on this revised text. We appreciate the changes to the first two paragraphs, which we support. These changes seem to us to improve the text. Likewise, we welcome the changes made by the Chair to paragraph 4. We are carefully studying your proposed changes to paragraph 6, its subparagraphs and the Roman numerals. Changes made to subparagraph A and B seem to be good.

However, on.

Subparagraph C and the listed limitations, we continue to be reluctant, especially with regard to Roman numeral 3 and 5, which we would suggest deleting. Our concerns are in line with those explained by the delegations of Germany and France. We look forward to continuing our discussions on these important issues.

Thank you, Chair.


Chair [2:25:55]:

Let me thank the distinguished representative of Denmark for sharing her comments with delegations. Next on my list is Israel, to be followed by the Russian Federation and Italy. But let me first pass the floor to the distinguished representative of Israel. So you have the floor.


Israel [2:26:10]:

Good morning to everyone and to you. I just put down the sign because there was good discussion this morning, good comments. We would prefer to improve our comments, taking into account what has been said. So if it's okay with you and we will continue in the afternoon, we'll prefer to wait.


Chair [2:26:27]:

Thank you. Of course, even better, because that would make it all more interactive and that is something I would like to promote. So let me now then pass the floor to the Russian Federation. So you have the floor.


Russian Federation [2:26:42]:

Thank you very much, Chair. I'd like to join other colleagues who expressed their gratitude to you and your team for preparing the new draft of box 3 of this document. We'd like to share with you some of our comments on the paragraphs in this box. More specifically, paragraph 1. We believe that we need to amend the language here in order to bring it in line with the previous language that we find in the different reports of the GGE. We believe that It could read as follows, that is close to the language of paragraph 17h of the 2019 report.

"Loss must not be used if it is of a nature to cause superfluous injury and/or unnecessary suffering, or if it is inherently indiscriminate, or is otherwise incapable of being used in accordance with the requirements and principles of IHL.".

As you can see, this language is very similar to what we have right now in paragraph 1, and we believe that we should stick to the language that we agreed to earlier. We'll say it once again, as we said earlier. From a substantive point of view, we do not see any difference here from paragraph 17h, and we do not understand why can't we simply take paragraph 17h from the 2019 report in the way it was presented. On paragraph 2, We find it necessary to have a general reference to international humanitarian law without separating separate principles. And this means that the language could be as follows.

Laws may only be used in accordance with IHL, including principles of distinction, proportionality, and precaution in attack.

On paragraph 3, here we see the need to strictly stick to the language of Articles 51 and 52 of AP I to the 49 Geneva Conventions. So the language could be as follows.

The civilian population as such, as well as individual civilians or civilian objects, should not be the object of attack, including by laws.

Now in paragraph 4. we find that it would be correct to stick to the understandings that the group achieved or reached in 2019 and reflected in paragraph 23a of the 2019 report, where we talk about the different stages of the life cycle. In particular, this paragraph could be reproduced in its entirety and would read as follows.

During the design, development, testing and deployment of weapons systems of loss, the risks, inter alia, of civilian casualties, as well as precautions to help minimize the risk of incidental loss of life, injuries to civilians and damage to civilian objects must be considered. Other types of risks should be considered as appropriate, including but not limited to the risk of unintended engagements, risk of loss of control of the system, risk of proliferation, and risk of acquisition by terrorist groups. We believe that.

This language helps us resolve the problem that is set in paragraph 4, the objective of reducing risks, the possible consequences of the use of such weapon systems. We find that the 2019 language is much broader than what we have currently in paragraph 4. It includes some very specific details and refers to certain specific risks that could be reduced through the relevant steps or actions. And we see the need to take the language and include it in the document that you presented, Chair. That was the objective. This language has been agreed to. We don't see any problems. And we do not see why that language cannot be included in this document. And the same applies to paragraph 5. First, we would like to recall that we have paragraph 5 in section or box 2. Paragraph 5 of section 2 refers to the same question, namely human control. It clearly states why we need human control. We believe that here there's a certain overlap between or duplication between paragraphs 5 in sections 2 and 3. The idea is the same, language is different, and this is something that my delegation, well, it creates somewhat a certain level of confusion. How do these paragraphs relate to one another? Because we talk about somewhat different things in both paragraphs, even though both paragraphs relate to the notion of human control. Moreover, Just like some other delegations said during our discussion, we would like to refer to paragraph 21C from the 2023 report, where we also are talking about control. And the idea that we find in paragraph 5 is reflected in that paragraph, and we do not understand why cannot we take the language from paragraph 21C of the 2023 report and use that language, especially since that language has already been agreed to, and from a point of view of substance, it doesn't differ in any particular way from paragraph 5 that you have included in your document. Moreover, We believe that it is fully justified to not limit ourselves to what is contained in paragraph 21C of the 2023 report. But we believe it will be appropriate to use the notions that we have in guiding principle C that also relate to the interaction between humans and machines. provide for a very important understanding in connection with such an interaction and this once again is language that has been agreed to and we do not understand why it has not been reflected so far in your document so this means paragraph 5 in light of the language that we have been referring to could be read out as follows.

Control with regard to LOS is needed to uphold compliance with IHL. Human-machine interaction, which may take various forms and be implemented at various stages of the life cycle of a weapon, should ensure that the potential use of LOS is in compliance with applicable international law, in particular IHL. In determining the quality and extent of human-machine interaction, a range of factors should be considered, including the operational context and the characteristics and capabilities of the weapon systems as a whole.

We believe that this is a very important point that really broadens in a very good way our understanding of what we mean under human control and human machine interaction. As we see, this gives us an opportunity to move forward on the question related to the language of the notion or term human control. On paragraph 6, first on the chapeau, in order to be consistent in the way we use different terms related to the question of the employment of such weapon systems and in light of the language that we already have in paragraph 2 in section 3. We believe that it is necessary to replace "can" with "may" and also We believe that we have to delete the reference to context-appropriate human control and judgment here in this paragraph, because we cannot simply include in every paragraph a reference to human control. We see that in every version the references to human control are increasing in a geometric progression. We have a reference to that in paragraph In box 2, we have a relevant paragraph in box 3 and the relevant paragraph in box 4. And in the new draft, we have a new reference to human control in section 3, paragraph 6. And we do not see any need to have this reference here. And we suggest that it be deleted on 6A. Our delegation still is not convinced that it is justified to follow the whole notion of predictability, reliability, traceability, or liability, says the speaker. From a legal point of view, we do not see any usefulness in the use of these notions. especially in the context of this document that some delegations believe will be a foundation for any type of instrument. And we believe that we simply have to delete 6A. On 6B, we're ready to support the language contained in the first sentence of the paragraph with the amendments that were introduced. However, the second sentence is something that we find superfluous because from a substantive point of view, it is fully covered by the first sentence and we believe it doesn't have any value added, especially in light of the amendments that we proposed earlier related to the notion of human control. Such precautions already do exist in the language and in the amendments that we are proposing, for example, in paragraph 5 of this section. And, for example, they are already contained in the guiding principle C. And guiding principle C is much broader than what we have here in the second sentence of subparagraph B. On 6C. As we said earlier, we do not find it necessary to have too much detailization of how human control should be carried out and, moreover, how specific measures related to human control should be exercised, related to limitations on the type of targets, length and geographical scope, duration, geographical scope and the scale of the operation. The very notion that such limitations have to be introduced is already a form of human control. Any further detailization will simply lead to serious problems in the future, especially related to the specific features of the functioning of such systems at the national level. And every state has their own ways for the functioning of the armed forces or internal regulations, internal national legislation. And all of this should be taken into account. And this is why we do not believe that it is justified to have such a detailed explanation of human control. And this is why we suggest that subparagraphs 2 through 6 to be deleted in 6C, especially as we said earlier, Very often we just see this to be a duplication with what is already said in previous paragraphs. Now on paragraph 7, this also represents a certain form of detailing on how human control should be exercised related to the critical functions. We believe that such detailization is superfluous. States are responsible for ensuring human control and defining specific measures of such control, something that should be done by countries themselves. Thank you.


Chair [2:42:19]:

I would like to thank the distinguished representative of the Russian Federation for sharing with us his proposals with regard to the text. I have one question, perhaps I missed it or I misunderstood it. With regard to paragraph 6, are you proposing to change "can" into "may"? I just picked something up with regard to replacing something by "may", but I...


Russian Federation [2:42:45]:

Okay.


Chair [2:42:46]:

Thank you for providing me with that clarity. Next on my list is the distinguished representative of Italy, and that's it before lunch, I think. Madam, you have the floor.


Italy [2:43:01]:

Thank you, Chair, for all the work you and your team conducted on this section that is particularly complex, as we see. We support generally the new rewording of the text, which appears balanced and better clarifies the reference to IHL and the responsible use of systems that feature artificial intelligence, such as LOAs. Regarding paragraph 4, this delegation is pleased to see the inclusion of the proposed suggestion by us and other distinguished delegations to replace deploy with develop, which extends the scope of prohibitions to the initial stages of development of laws, in line with the provisions of Article 36 of the first additional protocol to the Geneva Conventions, and therefore with a view to a full application of international humanitarian law. We listened carefully to concerns expressed by some distinguished delegations in terms of this provision possibly hampering technological development, but in our view, this proposal, rather than limiting technological development in itself, introduces secure guidance to it. It is understood that the primary objective of this document as a whole is to acquire the necessary consensus while remaining within the limits of the mandate of this group. We will continue to work towards that. It is clear to us How paragraph 6A of the text in introducing new terminology, predictable, traceable and explainable, adds to the document new elements of evaluation suitable for laying the foundation for a future instrument. We are aware, in fact, that these terms do not currently exist in international humanitarian law. But as highlighted also by other distinguished delegations, very eloquently by the delegation of Pakistan, we are facing an evolving character of warfare. And law shall try to respond to these new challenges. Also, the explicit reference to an operational element, such as the chain of command and control in paragraph 6B, implies compliance to IHL, even in the operational phases for the protection of those that are not involved in hostilities, and we fully support that. Finally, we are pleased to see the new rewording in paragraph 6C, roman 3, with the addition of "autonomously," which allows the purpose of the text to be preserved while adhering to operational needs and IHL requirements. And I thank you, Chair.


Chair [2:46:05]:

Let me thank the distinguished representative of Italy for sharing the views of her delegation and making proposals. And let me thank all of you for this morning. I think we had a very good discussion. And in that discussion, clear also that still a lot of work has to be done, but also we still have a lot of work to be in the afternoon because we have a number of delegations still on the list for this afternoon and I can imagine that some of you would also like to react to previous statements as already was announced for instance by Australia and I intend to to offer that opportunity. But for now, the meeting stands adjourned. And see you at 3 o'clock.