5th Meeting - 1st Session Group of Governmental Experts on Lethal Autonomous Weapons Systems 2025

Good morning, Excellencies, distinguished delegates. I would like to resume our discussion. I hope you all had a good evening and a good rest. We still have a lot to do. We were still discussing box number three yesterday, and we had at the speakers list at that moment when we stopped our discussions. We still had Norway. I see Norway is in the chair. So I would like to invite the distinguished representative of Norway to take the floor.

Thank you. Thank you, Chair. On a general note, box three is important to us as well. This is where we have the two tiered approach, prohibitions and regulations. And where we've started fleshing out these various limits and controls and other measures that are central to ensure that autonomous weapons remain within legal and ethical bounds. As we see it, this box has sort of two distinct sets of paragraphs, one on 24 on one hand and five through seven on the other. The first four, as others have said, mainly reproduce previous understandings based on well-known basic job obligations. We don't believe that they offer very much, apart from underscoring that these obligations also apply to the use of laws, yet they set the scene in a sense, so they have a place there. We just need to make sure that we don't inadvertently misrepresent IGL in some detrimental way once we start making these into the kinds of recommendations that our mandate calls for. And we would support the proposal to include rules, so that they would read rules, principles, and requirements, or some variation. of that. And on the issue of inserting development news, we agree with the sentiment because for laws, the developmental stage is probably even more important than for classical, classic weapon systems in terms of ensuring the ability to comply with The best way to address this would, in our opinion, be to formulate positive obligations relating to the development phase of laws, but we would also be interested in exploring this further, the proposal to add development to paragraphs 1, 2, and 4. So that would also entail that they no longer merely restate previous understandings of IHL, but our mandate is, of course, not just to restate the law, but to propose elements for a prospective instrument. On the second set of paragraphs, numbers five through seven, this is perhaps where box three adds the most value. Paragraph five is important to us. This builds on the understanding in paragraph one, but adds value by introducing the element of human control. If you can't exercise context-appropriate human control over a weapon or a weapon system, you will not be able to use it in compliance with IHL. And if you can't use it in compliance with IHL, well, then you can't use it. It's prohibited. So we appreciate that this prohibition linked to human control still has a ways to go in terms of maturity and common understanding, and that it would not yet be ripe for inclusion in a treaty or an instrument. However, as a mere proposal from this GGE for a possible element for some future instrument of an unspecified nature, we believe that it gives us sufficient guidance and direction. In a sense, it's not too different also from some of the cardinal principles of IGEL, which were themselves not entirely clear at the time that they were introduced, but fleshed out over time. Um, on the point also that was made yesterday that paragraph five substantively corresponds with guiding principle C and that guiding principle C is broader, this is, of course, to some extent true, but there is a fundamental difference. Guiding principle C only presupposes that there might be some human machine interaction. It doesn't outright require some level of such. And furthermore, it only suggests that this potential human machine interaction should ensure that the weapon system is in compliance with IHL. So it does not say that control is necessary to ensure compliance. Outright, that's guiding principle C. Now over to paragraph six. On a general note, we believe you've managed to capture much of the essence of the regulations or positive obligations that our discussions have converged around over the years. As for the chapeau, we would prefer, like others, to frame the issue in clearer terms and not have this variant that reflects unspecified previous items, and then to just show clearly that those elements which follow are intended to help ensure that the use of autonomous weapons systems is lawful, preferably then by drawing a direct link to human control and judgment. For instance, it could read, "To ensure context-appropriate human control and judgment with regard to the use and effects of laws and support compliance with IHL, parties to a conflict need to," and so forth. On to paragraph 6A, for us, this, uh, is, uh, an important paragraph, the concepts of reliability and predictability have been fleshed out, uh, quite a lot in this GGE over the years, including the important role that those properties play in ensuring meaningful human control and compliance with IGEL, we think they definitely merit being retained in box three. As regards traceability and explainability, we're familiar with those terms from the broader discussions of ethical use of AI, but they're more of a novelty maybe here in this GGE, and their relation to human control and agile compliance is less clear to us, as others as well. However, they are important for purposes of ensuring human responsibility and accountability, as well as to mitigate risks. So we believe they merit retention in the rolling text as such. If we are to keep them in this agile, human control related context, however, it would be good to understand that relation better. And finally, we think that the qualifier adequately here is important since it underscores their context dependent nature. Paragraph 6B, we think that the first sentence here would fit better in the chapeau somehow. Regarding the second sentence, we're a bit on the fence, but at any rate, we believe, like others, that moral should be replaced with ethical principles, which is a more familiar concept in this GGE. As regards 6C, those are important limitations and controls to mitigate risks and unlawful and otherwise deeply problematic effects of use, as well as to ensure that laws remain under effective human control. Like some others, we propose to add "as appropriate" at the end of the chapeau in order to underscore that the elements which follow in the subparagraphs with Roman numerals are non-exhaustive examples, the relevance of which depends on the context. That could possibly also help to allay the concerns of the various delegations that have expressed reservations about the content in subparagraphs 1 to 5. Regarding subparagraph 3, So, Roman III, we understand the rationale to be that limiting the number of engagements that a system can undertake reduces the potential consequences of errors in the system, as well as to promote a higher degree of human control through the need for some kind of periodic reactivation with the exercise of renewed context appropriate judgment and control that that entails. We would like to keep this paragraph, especially when it's rendered, if it's rendered clearly in the chapeau that the list is suggestive, not prescriptive. And we think also that the US proposal to add autonomously at the end could be useful to highlight the point about human reactivation. Roman IV is a helpful risk reducing measure. The more civilians that there are in the area of operation, the higher is the general risk to civilians. As some have pointed out, this might not always be a relevant limitation, uh, that problem is solved by the word avoid, which clarifies that this is not a blanket prohibition, um, right, and Roman 5, um… relates to the targeting of people as we read it, as opposed to objects. And we think that this has a place in our text, as the targeting of people raises much more difficult legal and ethical problems than targeting of inanimate objects, and that that calls for special consideration. The way that it's phrased now, we understand this as a call to consider not using laws against personnel and not a prohibition as such against anti-personnel laws. in line with what we understood the Israeli mission to propose yesterday, we think that the list could also include a sixth limitation that relates to the geographical scope of the system, uh, reading something like limiting the operation of laws to defined geographical area. And, uh, at last, paragraph seven, um, the term, uh, mission parameters, uh, I mean, it's not entirely clear to us. I mean, the idea as we understand it is that we need to ensure that the system is not able to autonomously change its mission or critical instructions. In other words, to ensure that the link between the command intent and judgment and then the operation of the system, that that link remains unbroken. And this is something that we agree with, but like others, we believe that this should be clarified. Thank you.

Let me thank the distinguished representative of Norway for sharing the views of his delegation, for making proposals and, indeed, also giving some background on certain issues. Very helpful. Next on my list is Turkey, to be followed by Sri Lanka. But let me first pass the floor to Turkey. You have the floor.

Thank you, Mr. Chair. The first four paragraphs are acceptable as they reflect IGL requirements. We support the US proposal in paragraph 3 concerning the protection of individual civilians. We propose revising the chapeau of paragraph 6 to read, "Parties to a conflict should take necessary measures to," and taking paragraph 6 into account under box 4. Additionally, the second sentence in paragraph 6B appears superfluous, and we request the addition of the Roman numeral 3 in paragraph 6C. I thank you.

Let me thank the distinguished representative of Turkey for his comments and proposals. Next on my list is Sri Lanka. You have the floor, madam.

Thank you, Mr. Chair. As mentioned in my previous intervention, Sri Lanka reiterates its proposal to use the term "develop and use" in paras 1, 2, 4, and 5 in the box 3, on the basis that both development and use phases are crucial in the prohibition of lethal autonomous weapon systems, as the development phase sets the foundation for the weapons' capabilities and limitations, and ensures its compliance with IHL in the human control in the entire life cycle of the weapon. In order to ensure effective prohibition, it is necessary to cover both development and use phases. Weapons that are developed without the capability to comply with key principles of IHL, including distinctions, proportionality, and precautions are unlawful and must be prohibited. It is also noted that other arms control treaties, including but not limited to the Anti-Personal Mine Ban Convention, the Cluster Munitions Convention, and the Chemical Weapons Convention, et cetera, also prohibit both the development and use of the respective weapons. The matter of laws is crucial and complex than the conventional arms. My delegation also listened carefully to the views of the delegations on the human control and judgment. Sri Lanka opposes the deletion of the term human control from the text, and we are in favor of retaining both terms, human control and judgment. Thank you, Mr. Chair.

Let me thank the distinguished representative of Sri Lanka for sharing her views and for coming with suggestions. Looking at the list, I see no further requests. Yes, I see a black. Ireland, and there's another one on the left. Switzerland. But first, Ireland, you have the floor.

Thank you, Mr. Chair. We very much welcome the healthy debate on this section, and we just want to feed some points into the debate that came up both yesterday and this morning. As an overall point, we want to echo some points made yesterday regarding our work in general. We heard some delegations prefer to stick to consensus language from 2019 or 2023. But we have a new progressive mandate that is designed to allow us to build on our previous work. And the text we have in front of us Here is indeed that, a combination of previous agreements, existing protocols, as well as novel concepts, which are necessary due to the novel nature of the rapidly evolving weapon systems under consideration. And we thank you, Chair, for such a strong text in that regard. Further, we reiterate that IHL is a floor and not a ceiling. Indeed, we are grappling with how IHL applies to autonomous weapon systems. in some cases, we will need to further develop IHL principles in order for us to ensure that humans remain accountable over the use of force, given the unique nature of these weapon systems. In this vein, we want to offer some clarifications on what we see as context-appropriate, meaningful human control and judgment, which some delegations saw as vague or undefined. For us, this term refers to the requirement that human operators retain sufficient oversight and decision-making authority over autonomous weapon systems to ensure that their actions align with ethical, legal, and military objectives. It is meant to prevent these systems from making life-and-death decisions without human judgment, allowing for accountability under the law. This can be achieved through setting geospatial and temporal limits as defined under paragraph 6. In order to ensure accountability, the weapon systems also need to be predictable, reliable, traceable, and explainable, as we described yesterday. The word meaningful is also intended to close the loophole presented by nominal human input, which is a situation where an operator provides minimal or superficial oversight over the system's actions without genuinely influencing its decision-making process. This can include, for example, rubber-stamping target selection processes using AI or sensor data. This type of human intervention does not satisfy the requirements to ensure human accountability and control over the use of force, and it can also lead to automation bias, which we will touch upon further in box four. For these reasons, it is important that elements under 6 remain in the prohibitions and restrictions section, for they are the necessary elements that allow for meaningful human control and for human accountability in accordance with IHL. And for this reason, we would like to keep the need to in the chapeau of paragraph 6. Yesterday, I believe the U.S. representative referred to weapons that currently exist that operate without human control that are not prohibited, such as perhaps precision-guided munitions. These types of munitions could fall under the regulation of the two-tier approach, as they fulfill the geospatial and temporal limitations as contained under 6C. Otherwise, they could be included in the exclusion paragraph under box 1. But we do not agree with the suggestion by some delegations to remove human control from the text. And briefly, just reacting to some of the proposals from yesterday, we welcome Sweden's proposed formulation for the Martens Clause paragraph, which seems promising. And in 6B, we also join others in suggesting ethical considerations instead of morality. Thank you, Chair.

Let me thank the distinguished representative of Ireland for sharing his views and also putting this discussion on box 3 in a wider context. It has already been done by others. I think it was also Canada that referred to accountability and responsibility. We are going to address in box 5. Next on my list is Switzerland, to be followed by the EU. But let me first pass the floor to the distinguished representative of Switzerland. You have the floor, sir.

Thank you, Mr. Chair, and thank you for giving me the floor again. We heard many useful proposals over the last hours and yesterday that we are studying closely, but which we're confident will bring us closer to moving forward. Let me just share a few considerations on paragraph six. We, for instance, wonder whether the chapeau, as it stands, is absolutely necessary. For instance, like in other instruments, including protocols of the CCW, it might be easier to just define the measures that states agree are necessary with regard to autonomous weapons systems, be it for compliance with IHL, ensuring human control, or for other considerations. The following subparagraphs would then simply begin with "States and parties undertake to ensure" and so on. In any case, we feel that the chapeau as it stands now is too restrictive, as it only applies to parties, but certain measures are also relevant at the development stage. This is why we would propose to aim the measure at states and parties to a conflict. With regard to paragraph 6b, We see the added value of moral in there, and as you explained already why it is there, because the reference to nominal human input has been removed, and we also understand that the notion is not on the same level perhaps as legal. On the other hand, we understand that it's important to not give the impression that human control is only performed through nominal human input. So we are open to look for fixes, and we heard that ethical considerations could be an option. And perhaps we can also look at language that speaks more about procedures for trained human operators to activate or deactivate functions in weapon systems so as to enhance control or improve decision-making over the use of force. With regard to the chapeau of paragraph 6, we heard that some called for adaptations to include as appropriate. It would seem necessary to us to perhaps make this a little bit more focused, for instance, by stating that it needs to be appropriate in the specific context and by making the chapeau not exhaustive, for instance, by including inter alia or a similar term. With regard to paragraph 6C.1 on the activation, we are of the view that the letter 1 and two could be combined in order to make it an alternative, ensuring that laws can be timely influenced by a human user after activation, which includes deactivation, would be an option, or that laws incorporate self-destruct, self-deactivation, or self-neutralization mechanisms. On the third sub-paragraph, on limiting the number of engagements. We believe this is an important approach, but we also hear the many concerns. We heard yesterday that, I think if we understood the US delegation correctly, that this could also be a form of a precautionary measure, and this is something we would share its precautionary measure, which limits eventually the scale of an operation, at least before further human guidance is required. This is particularly important if initial errors of judgment have resulted in inappropriate or unlawful targeting, but it can also be a possible measure helping to implement the precautionary requirement to cancel or suspend an attack if it becomes apparent that the objective is not a military objective, especially protected, or that the attack would be disproportionate. With regard to paragraph 7 on mission parameters, we see and support the idea behind it. But we agree with some of the skepticism that we heard in different statements. It's not all the mission parameters that are of relevance. It's relevant if a system makes changes to parameters linked to the core functions we are discussing and focusing on in our deliberations. So we understand that it should not include technical adjustments such as altering flight paths, which do not necessarily pose a challenge to IHL. So we see multiple options. We can make it clear that it's only modification with legal or humanitarian implications that are problematic. or that it's ensured that critical mission parameters can only be modified with the necessary human intervention, or that they cannot be modified outside of the operational framework defined by the chain of human command and control. So basically, the main focus of the paragraph in our view should be that it's the critical functions are changed, in particular identifying, selecting or engaging targets. Thank you so much.

Let me thank the distinguished representative of Switzerland for further elaborating the position of his delegation and for making suggestions. Next on my list is the European Union, to be followed by Pakistan. But let me first pass the floor to the distinguished representative of the European Union. You have the floor.

Thank you, Mr. Chair. I would like to make a few comments on elements presented in this box, and specifically on regulation part in para 6. The European Union thinks that the future set of elements of an instrument and other possible measures should include the following possible requirements to ensure that laws are being used in accordance with international law, in particular IHL and relevant ethical consideration. First, in the study development, acquisition or adoption of a new weapon, means or method of warfare, determination must be made whether its employment would, in some or all circumstances, be prohibited by international law. Second, ensure appropriate levels of human control and judgment, which must be retained during the entire life cycle of the weapon system; third, control and/or limit the types of targets that the system can engage, as well as the duration, geographical scope, and scale of the operation of such weapon systems, such as, where appropriate, incorporation of self-destruct, self-deactivation, or self-neutralization mechanism into munitions or the system. Fourth, prevent and mitigate risks of unintended bias, including their potential impact on determining combatants and other risks regarding the safety and security of such weapon systems. including the risk of diversion to unauthorized users, including terrorist groups. And lastly, ensure the highest possible and appropriate level of reliability, as well as the traceability of such weapons systems in order to securing commanders and operators' responsibility and accountability. Thank you, Mr. Chair.

Let me thank the distinguished representative of the European Union for sharing her comments. Next on my list is Pakistan. So you have the floor.

Thank you, Mr. Chair. Thank you for giving me the floor for the second time. This, I hope, will be our last intervention in this box. We would like to respond to some of the questions that were raised on how the principles of predictability, reliability, traceability, and explainability will be implemented and by whom. What we wish to clarify here is that we are not seeking to impose a rigid technical blueprint on how these principles will be realized. So the questions, the subset questions that follow each of these principle is not part of the discussion as such. Different states will determine how best to implement them in accordance with their national processes, legal frameworks, and technological capabilities. However, what we are trying to do is to firmly root these principles of predictability, reliability, et cetera, within the framework of IHL, because without them, IHL compliance simply cannot be assured. So in that sense, these principles will help the states in their respective national domains to implement their legal frameworks and technological capabilities. And I think this is a concept which we would request our colleagues to have some examination. The question of predictability and reliability, Mr. Chair, ensures that the system will behave in a manner that is foreseeable, stable, and consistent with IHL obligation. A system that cannot reliably distinguish between lawful and unlawful targets or that behaves erratically under battlefield cannot meet the fundamental requirements of distinction and proportionality. Similarly, traceability is essential for as it documents and reviewable, making accountability and investigation process possible. Without traceability, there is no way to verify why a system engaged a target and who's responsible and whether any IHL violation occurred. Explainability ensures that human operators and commanders can understand the system's reasoning and actions, allowing them to exercise meaningful oversight. If a system's logic is completely opaque, then oversight is reduced to a formality rather than a safeguard. These principles, Mr. Chair, are not abstract or optional technical refinements. They are practical enablers of distinction, proportionality, and precaution, which is the very foundation of IHL. weapon system that lacks these characteristics risks operating in a manner that is unpredictable, unaccountable, and ultimately incompatible with IHL. And when we view it with the concept of the bias in military AI, which we will discuss in subsequent chapter, the importance of these principle reinforces itself further. In this context, we found the intervention of our UK colleague yesterday to be particularly helpful. As we understand, they too highlighted the importance of rooting these principles within IHL, as doing so provides clarity and ensures that they serve as meaningful safeguards rather than vague aspirations. We have also tried to capture this in our proposal of five bis. linking these with IHL. Mr. Chair, we also struggle with the argument that since the concepts, these principles are not explicitly contained in IHL, they should not be part of our conversations. This reasoning is inconsistent with how we have already-- we have always approached our work in this CCW framework. Our role here is just not just to restate existing IHL. rules, but also to clarify, refine, and where necessary, expand their application in response to evolving technologies and new challenges. This is precisely what we have done with past weapons systems, and it is precisely what we must do now. The body of IHL must continue to evolve with the evolution of technology where necessary. The real question then, Mr. Chair, is is not whether the states would implement these principles, but how we ensure their implementation in a way that guarantees compliance with the IHL and help the states to formulate national processes within these principles. This is the discussion that we must factor in Box 2, and we look forward to a constructive engagement on these issues. Thank you.

Let me thank the distinguished representative of Pakistan for further explaining his position and also reacting to the proposals made by other delegations. He was the last on the list, but I see Ukraine, the distinguished representative of Ukraine. So you have the floor.

Thank you, Mr. Chairman. I'm sorry, I will have only short remark regarding last sentence of Pakistan delegation. I should be taking to account that if you consider it prediction on forecasting effects of using LAFs, we should be taking to account also measurement of enemy troops. If enemy troops use electronic warfare, means spoofing, air defense system, effects of using particular UAV can be not predictable. Thank you.

Let me thank the distinguished representative of Ukraine for also highlighting some impacts of technology on the battlefield at this moment. As said before, This is going to be a bit like an auction, one, two, three. Is there anybody else asking for the floor? If not, then I think we have had a very, very thorough discussion indeed with regards to box number three. And it's not easy to summarize, but let me nevertheless try. And let me start by, um, again, recalling that there is a very close link to box number four, some of you have also made proposals to shift some of the text to box four, so we will in a moment start our discussion with, on box number four. Um, we have seen, uh, comments from the floor, you could put it, If you look at the text, it's partly in three instances, in boxes one to four, as was described by a number of delegations, reflecting existing international humanitarian law, building, I say building, on text agreed earlier. We got some comments on that, that it is not identical to text from 2019 and 2023. That's correct. There were some reactions to remarks on that also from the floor stating, reminding us of our mandate. We are now drafting possible elements. Of course, if we have new text, then indeed we should also get consensus on that text. But the automatic reaction from the Chair is not always to take the old text, because otherwise we could have just made a kind of a compendium, and we are now trying to make a set of elements. In box, in para 5, we have, of course, the issue where we, with regard to human control, we have there context appropriate human control and judgment. Many delegations have reflected on that. That has given us a pretty good insight on how different delegations think about that. We are still looking for a magic formula, but we will try to do some work on that, and we will continue engaging also informally with delegations. With regard to the paras and over the text, we have seen some remarks with regard to use, developing use, so there is quite some delegations who commented on that. Uh, with regard to box, uh, with regard to power six and seven, um, we had numerous proposals, but I think, um, we had also some very interesting suggestions how we might restructure the text, uh, in a way that it becomes palatable for more delegations on the floor, and we will carefully look into that. Also with regard to the chapeau, and then of course, we also will have to decide on whether it still has to remain here or it has to move to box 4. We have some concrete proposals for that also. With regard to the power 7, Some suggestions were made how to make that work. But of course, we also have there again context of proper human control and judgment, which is a topic still under discussion. So that issue that first came up at the end of box 2 is of course still something to be resolved. I think this is more or less what I would like to say about what we have done here. Let me also recall, of course, some of the remarks that what we have here as text has to in a way ensure also the accountability and responsibility of the humans operating the systems, but also of the states responsible for the humans and the systems, or as some people call it, the responsible chain of command. I think that underlines already that in a way, we have something in front of us that looks like a set of elements, because this has effects through the text. We still have no headings. I know some of you would like it, others don't like it. For the time being, I'm not creating an extra problem, so I'm not going to put them in. I think this is more or less what I can say about it. I'm just looking. Yep, okay. Always important. I could have a blind spot somewhere. Also, with the lack of oxygen in this room, it's not always possible to stay totally focused for six hours a day. Then I would like to move to box number four. And we have received already from a number of delegations also written comments. Some of you have already referred to text there in more or less general remarks. And I would now like to open the floor to hear your comments on the text here in box number four, the paragraphs 1 to 8. I recognise the distinguished representative of Australia. So you have the floor.

Chair, thank you for giving me the floor. Firstly, Australia supports the inclusion of paragraphs 1 to 4 in this box, which addresses the legal review of laws to ensure compliance with international humanitarian law. We believe strengthening compliance with existing IHL, including through Article 36 legal reviews, is the most effective way to increase confidence and to manage legal risks that are associated with this new type of weapon system. This GGE has also recognised over a number of years the value of the exchange of relevant best practice in undertaking legal reviews. In this respect, Australia has been pleased to host expert meetings on the legal reviews over the last two years. I would note that The third expert meeting on the legal review of autonomous weapons systems is scheduled to occur next week here on Monday and Tuesday at the Palais. Whilst we've been very pleased with the positive response, we still welcome attendance and active engagement by delegations and are very happy to provide further information about next week's meeting if necessary. In respect to paragraphs 6 and 7, we propose certain edits to achieve greater clarity on data sets and bias. Firstly, we consider periodic assessment may be understood in different ways and may be understood to have different meanings. We think the intention of this paragraph could be better captured by replacing the words "periodic assessment" with, and I quote, "testing and evaluation, including at appropriate points through its life cycle." So I repeat, "testing and evaluation, including at appropriate points throughout its life cycle," which we already see in the current rolling text at paragraph 5. We consider that testing evaluation provides greater detail as to what is required and at the appropriate points through the life cycle makes it clear that such testing and evaluation is not limited to a specific point in time, but rather should occur when appropriate for the specific technology. We're also open to the ICRC's proposal to insert mitigate here. Chair, we also consider we should use more objective language regarding assessing bias. With that in mind, Australia believes that unintended bias may be better suited than harmful bias. We consider the term harmful bias as being inherently subjective. and difficult to distinguish between permissible harmful action against military objectives. Clearly, any lawful attack against a military objective is inherently harmful and therefore impractical in implementation. We recommend unintended bias which is readily distinguished from intended permissible bias against military objectives. As such, it supports implementation of an objective standard and criteria. In our view, unintended bias provides clearer, practical guidance for robust test and evaluation and monitoring measures than can detect and correct such unintended bias. We also wish to ensure there are not differing measures concerning bias by addressing bias in one paragraph, therefore deleting paragraph 7. Having two different paragraphs relating to bias, data and bias, I beg your pardon, with different standards may create confusion with respect to different requirements correction. Thank you, Chair.

Let me thank the distinguished representative of Australia for his comments and suggestions, and let me also again draw your attention to the workshop on Monday and Tuesday with regard to legal reviews. I would highly recommend it. Next on my list is the Russian Federation, to be followed by Germany, and then we have the Asia-Pacific Institute for Law and Security. But let me first pass the floor to the distinguished representative of the Russian Federation. Sir, you have the floor.

Thank you, distinguished Chair. With respect to Box 4, Russia has no major problems with the sort of operational part of that section. We share the understandings that were put forward in paragraphs 1 to 8. However, we do have difficulties linked to the fact that a number of paragraphs that are in this box duplicate each other, either duplicate paragraphs that were in previous sections, specifically in terms of content. In particular, paragraph 2, which talks about the need to ensure meaningful human control, duplicates a similar understanding that we had earlier in sections 2 and 3. In our opinion, adding in each section an understanding that human control must be duly guaranteed would be redundant. I think that we can delete paragraph 2 without in any way affecting the rest of the content of the document. As to paragraph 4. which talks of conducting legal reviews and licensing of such weapons systems. That paragraph also duplicates the provisions of paragraph 3, which talked about the need to conduct such assessments at the pre- application pre-deployment of such weapon systems. We referred, when we made our comments on section 3, especially on para 5, to the guiding principle C, which concerns specifically the issue of conducting those kinds of reviews, compliance or taking into account relevant risks or concerns in the context of the further operation of such systems. Therefore, I think that paragraph 4 seems redundant here, and we could fully delete it without in any way harming the content of the document. Again, we stress that the understanding regarding conducting such reviews is already present in the document. On para 7, I think that it is redundant in view of what we already have in para 6. Para 6 already talks about relevant problems in data, harmful bias in data sets. Accordingly, without affecting the content of section 4, I think we can just delete para 7. It's already fully covered by what you have in paragraph 6. Besides that, our delegation continues to adhere to the position closely regarding that it would not be wise to use wording that carries a subjective nature and not a legal nature. In particular, paragraph 3, we don't think it's necessary to use the word "properly" before "understood" because "properly" serves as a qualifier with respect to that paragraph and could serve as a grounds for various interpretations of understanding what that of what is contained in that paragraph so we don't see any need for that qualifier properly we think the relevant potential and the relevant aspects related to laws should be clearly understood. And therefore, again, we don't need that word properly. Then para five. I think it would be necessary. To speak overall of conducting testing and assessment with respect to relevant evaluation of systems. Overall, not just in specific realistic operational environments. Moreover, we don't deny that such testing and evaluation could be conducted in that realistic operational environment. Speaking specifically, we could add the words "including" after "evaluation". That might settle the problem and give paragraph 5 a more general and comprehensive nature, so that it would concern overall issues of testing and evaluation of the functions of such systems, including in realistic simulated operational environments. I thank you.

Let me thank the distinguished representative of the Russian Federation for sharing his comments with us. Next on my list is Germany, apt to be followed by the, what was it, Asia Pacific Institute, the RSIS, and Japan. But let me first pass the floor to the distinguished representative of Germany. You have the floor.

Mr. Chair, Germany attaches great importance to ensuring that the framework of human control is in place during the complete life cycle of a weapon system. We therefore welcome part four of the rolling text, highlighting inter alia that states should ensure that laws are operated with context appropriate human control and judgment across the entire life cycle of the weapon systems in para two. We particularly appreciate that weapon systems under development or modification are explicitly mentioned in para 4, as these parts of the life cycle will, in our view, play a significant role in AI-enabled systems. We share the view that, as mentioned in para 4, reviews are important to ensure compliance with IHL. We also appreciate that the rolling text mentions harmful biases. That may be contained in data sets in para six of part four, and measures to address possible harmful bias in artificial intelligence capabilities relied upon by laws in para. Germany attaches particular importance to challenges that may be posed by unintended biases, inter alia on gender, racial, or ability related aspects, and we most welcome the fact that the discussions on this issue in the framework of the GGE have intensified in the past year. In that regard, I would like to draw your attention. of the, also the attention of the distinguished delegates to a side event on bias in autonomous weapon systems and compliance with international humanitarian law hosted by the Stockholm International Peace and Research Institute, SIPRI, which will take place today at 1:15 in room 21. We strongly support that para 8 of part 4 of the rolling text highlights the necessity of appropriate guidance, training, and instructions for human users. These aspects are an indispensable part of ensuring the responsible and IHL-compliant use of weapon systems featuring autonomy, as commanders and operators need to properly understand the system and its effects. Thank you.

Let me thank the distinguished representative of Germany for sharing her views and comments and making suggestions. Next on my list is the Asia Pacific Institute for Law and Security to be followed by the representative from RSIS and Japan, but first the representative of the Asia Pacific Institute, you have the floor.

Thank you, Chair. The Asia Pacific Institute for Law and Security, APLS, is a specialist organization that works inter alia on the development and universalization of good practices in relation to the legal review of new weapons. Therefore, we are pleased to see the inclusion of this box in the rolling text. We also appreciate the Chair's efforts to organize and consolidate earlier language on testing and reviews into a single coherent part of the group's working document. This consolidation highlights the different forms of human intervention that ought to take place in the development phase of weapons systems, including laws. We would, however, like to make a few suggestions in terms of the specific language. First, the chapeau of this box concludes with the word "should", indicating that the paragraphs that follow reflect desirable actions or good practices. However, the obligation to undertake legal reviews of new weapons, reflected in the first sentence of paragraph 1, and also in paragraph 4, constitutes a legal obligation, at least for those States that are party to Additional Protocol I. suggesting that legal reviews should be conducted risks diluting this obligation. Stronger language from the group's earlier consensus documents would be preferable here. For example, paragraph 23 of the 2023 report noted that in accordance with states' obligations under international law, a determination must be made as to whether a weapon's employment would be prohibited by international law. A textual fix might be to place paragraph 1 at the beginning of this box before the current chapeau and to use previous consensus language. Moving on to paragraph 4, which on our reading refers to a situation where a legal review needs to be updated or carried out again due to the modification to the weapon system. Determining what kind of changes should trigger this re-review is a complex legal and practical question. It is impractical to conduct a new review as a consequence of any minor modification, and not every modification to a weapon system would make it a new weapon for the purposes of paragraph one of the text before us. The words "significantly changes" elegantly captures this practical consideration. However, it would be helpful to hear from states what kind of changes they consider sufficiently significant to necessitate a re-review. Paragraph 4, however, is currently limited to the situation where there are changes to the use of an existing weapon system. It is certainly true that a change to the normal and anticipated use of a weapon may necessitate a new legal review. But especially with regard to laws, an equally significant issue is a change to the way in which the system operates, even where its use remains the same. We would suggest inserting the words "performance or" before the word "use". Also, the phrase "existing weapons systems" is repetitive and could be shortened. So, in summary, the paragraph could read: ensure that weapons systems under development or modification that significantly changes their performance or use are reviewed to ensure compliance with IHL. Furthermore, the group may be in a position to add more depth and granularity on the question of legal reviews. Some helpful draft language was proposed in 2023 that could be useful. For one, that draft language helpfully captured the idea that a legal review of laws may need to consider not just outright prohibitions of particular weapons, but more broadly, the capacity of the laws to be used in compliance with the principles and rules of IHL. And these include, as mentioned regularly in this group, the principle of distinction, the rule of proportionality, and the requirement to take precautionary measures. The 2023 draft language suggested that legal reviews should assess whether laws are "capable of being used in conformity with applicable international humanitarian law and international law." This additional language could be added to the rolling text as a new numbered paragraph, potentially after paragraph 5. Also, draft language from 2023 suggested that legal reviews of laws, quote, "should consider, inter alia, the technical performance, intended use, and the possible tasks and types of targets as appropriate," end quote. Elements of this language could add helpful detail to the rolling text, potentially as a new numbered paragraph. And finally, as a very minor point, in paragraph 8, The words "States should" are superfluous owing to the language of the chapeau. I thank you, Chair.

Let me thank the distinguished representative from the Asia-Pacific Institute for his comments and suggestions and also for reminding us of previous work done by the Group. Next on my list is the RSIS. Sir, you have the floor.

As this is our organization's first time taking the floor, I would like to express our appreciation to you and your team. The purpose of my intervention is to highlight an area for the group's consideration that we feel would strengthen the rolling text, namely to underscore the importance of testing, evaluation, validation, and verification, or TEVV. Although TEVV is already mentioned in paragraph 5 of box 4, the current phrasing is focused on enabling, and I quote, a human user to have a reliable expectation of how the weapon system will perform in the anticipated circumstances of its use." End quote. Chair, testing, evaluation, validation, and verification for laws takes on a significant layer of complexity when we consider AI-enabled systems. This is because the inclusion of AI-enabled functions and features in laws introduces difficulty in predicting the output of these systems, complicating TEVV assessments based on benchmarks. Reliability is an important objective motivating TEVV activities, but our argument is that beyond assessing reliable performance, TEVV can be employed to evaluate key attributes of laws, such as correctness, interpretability, bias, robustness, resilience, and uncertainty. TEVV also plays a crucial role in supporting the legal review process. The review obligation is outlined in the first sentence of paragraph one in box four. It draws on Article 36 of Additional Protocol I to the Geneva Conventions. However, there is currently an absence of clear evaluation standards for states to assess new weapons, means, and methods of warfare. Furthermore, while scenario-based performance testing is one possible TEEV methodology, There is a much larger toolkit, including red teaming, adversarial testing, and real-world field trials, each contributing uniquely to a holistic assessment of the system's capabilities and limitations. Regarding testing to address the threat from adversarial cyber attacks, this issue has not been addressed in detail during the discussions of the GGE thus far. There is considerable uncertainty regarding the capability of military AI systems to withstand adversarial cyberattacks. This underscores the necessity for TEVb processes related to laws to address this issue, particularly within the context of assessing reliability of AI-enabled laws. Moreover, enhancing the resilience of laws against adversarial attacks is integral to ensuring accountability. If a system is easily compromised, it might be challenging to attribute responsibility for its actions, raising serious concerns about compliance with international law. Therefore, incorporating adversarial testing into the broader framework of testing and evaluation is essential to reinforcing both the security and accountability of laws. Given the multifaceted role of TEVV, we would like to encourage delegations to consider expanding paragraph 5 to acknowledge these additional objectives and methodologies. This would provide a more accurate reflection of TEVV's role in the assessment and governance of laws. I thank you, Chair.

Let me thank the distinguished representative of RSIS for for elaborating the concept of TEVV. Next on my list is Japan, to be followed by Canada, Turkey, the European Union, the ICRC, the Republic of Korea, and Denmark. So I have quite a list at this moment. But first, the distinguished representative of Japan, to be followed by Canada.

Mr. Chair, good morning to all. In order to fulfill our national responsibilities, while maximizing the benefits of AI technology. Well, it is important for us to correctly understand the characteristics of AI technology and take necessary control measures in the process of promoting advanced autonomy. In other words, to reduce and avoid risks of unintended and inappropriate behavior of AI technology. So in this context, such a person that measures to minimize AI bias and monitoring of capabilities updated by self-learning and introduction of self-guarded to avoid unintended consequences are quite particularly important. So concerning this particular box of this wording text, we would like to continue careful consideration and discussion on context, approach, appropriate human control and judgment in paragraph two. But the other paragraph of this box are quite acceptable for my delegation at this moment. Thank you very much.

Let me thank the distinguished representative of Japan for sharing the views of his delegation and making proposals. Next on my list is Canada, to be followed by Turkey and the EU. But let me first pass now the floor to the distinguished representative of Canada. So you have the floor.

Thank you, Mr. Chair.

Good morning, colleagues.

My delegation would like to offer some comments on the paragraphs of box 4. Beginning with the chapeau, we would like to suggest using the term deployment rather than employment for consistency. On paragraph one, we recommend adding, well, for paragraph one for us, the first part of the paragraph is a must for us in terms of the legal obligations under Article 36. While the exchange of best practices would be a should for us. So perhaps we could see a formulation of separating those two. In paragraph two, we see this as repetition from previous sections, and we would suggest removing paragraph two. On paragraph four, at the start of the paragraph, we would suggest inserting autonomous or semi-autonomous functionality after the ensuring that so that the suggestion would be paragraph four would now read ensure that autonomous or semi-autonomous functionality of weapon systems under development and so on carrying on the rest of the paragraph. On paragraph five, we would suggest to insert consistent and reliable after a human user to have, and paragraph, to have consistent and reliable expectations of how the weapon system will perform and so on. On paragraph six, we have a few suggestions here. So at the beginning of this paragraph, We would suggest adding development -- develop an assessment regime based on comprehensive testing and evaluation, and so on. We would also suggest inserting "relied upon by laws" after "data sets" in that paragraph. We also support the Australian recommendation to remove "harmful" and replace with "unintended bias" in that paragraph. On paragraph 7, we also agree that this paragraph could be removed altogether or merged with the previous paragraph. On para 8, our suggestion here is to remove human users of LAWS and replace it with those in the responsible chain of command and operators of LAWS.

Thank you.

Let me thank the distinguished representative of Canada for making comments and suggestions. If we do not yet have your suggestions, we would appreciate to receive them, because I was not able to scribble everything down. Next on my list is Turkey, to be followed by EU and ICSC, but let me first pass the floor to the distinguished representative of Turkey. You have the floor, sir.

Thank you, Mr. Chair. We are amenable to incorporating the term "ensure" in this box, provided the SHPO specifies that states should undertake the necessary measures. This stipulation is crucial to maintain a balanced and flexible approach and to avoid prejudging the outcome of the GGE. I thank you.

Let me thank the distinguished representative of Turkey for the suggestion made. Next on my list is the EU, to be followed by ICRC and the Republic of Korea. But let me first pass the floor to the distinguished representative of the EU. Madam, you have the floor.

Mr. Chair. With regard to bias in algorithms and language proposed in para 6 and 7, in this section, the European Union would like to reiterate that measures to address bias and to mitigate risks associated with bias in algorithms used in laws could include the following: comprehensive testing and reviews to identify and correct potential biases; Training of operators. Rigorous documentation of data sets used in laws in conformity with national legislation. Testing of algorithmic models against benchmarks that evaluate their operation against potential bias. As appropriate, transparency in how data sets are acquired and handled in conformity with national legislation. And lastly, particular care and specific measures with regards to the integrity, veracity, and quality of data in conformity with national legislation. Thank you.

Let me thank the distinguished representative of the European Union for the proposals made. Next on my list is the ICSC. You have the floor.

Thank you, Chair, for giving us the floor. We commend the inclusion of this box, particularly paragraphs 1 to 4, which refer to measures that are linked to the obligation for states to conduct legal reviews. We see legal reviews as a critical measure to ensuring that the use of autonomous weapons will not be in violation of IHL and other applicable international law. Given the mandatory nature of this obligation under Article 36 of Additional Protocol I to the Geneva Conventions at least, we echo the comments already well made by the Asia-Pacific Institute for Law and Security for edits which change the language of "should" as is currently seen in the chapeau to "must". At the very least, this should apply to paragraphs 1, 3, and 4, though ideally it could apply for the remaining paragraphs. Looking at paragraph 4, this provision importantly captures the need to review weapon systems in cases of modification. However, as is set out in the commentary to Article 36 of Additional Protocol I, and as has already been articulated by APLS, this obligation is not limited to modifications that change the use of a weapon system. It also applies where there are changes to functions or possible effects. And so we would propose including those terms in paragraph four of this box. And we also note with interest the language of performance that was suggested here by APILS. Turning to paragraph six and seven, we welcome recognition of the need to take measures to address the different biases associated with the development and use of autonomous weapon systems. Though the issue of bias within a system or its data sets, sometimes called algorithmic bias, this is a different concept to human cognitive biases, such as automation bias, which can arise when humans interact with machines. And each of these will require different measures. So we've suggested separating paragraph six into two subparagraphs, with the first addressing bias in AI capabilities that are relied upon by laws, and the second referring to cognitive tendencies of humans that interact with those laws. We'd also support Australia's proposed changes that have been echoed by other delegations to replace "harmful" with "unintended", particularly when referring to that first type of algorithmic bias. Thank you, Chair.

Let me thank the distinguished representative of the ICRC for sharing her comments with the group. Next on my list is the Republic of Korea, to be followed by Denmark and Colombia. But let me first pass the floor to the distinguished representative of the Republic of Korea. You have the floor.

Thank you, Mr. Chair. As we reach the midpoint of our week-long meeting, my delegation would like to once again express our deep appreciation to you and your team for dedication and leadership in guiding our intensive discussions. With regard to box 4, which is generally acceptable to my delegation, we would like to raise three minor points. First, as many delegations have emphasized the importance of building upon past GGE discussions, my delegation believes that it is widely shared understanding within our group that our work should not hinder technological progress for peaceful purposes. Accordingly, we propose incorporating language from Guiding Principle J into the chapeau of this section. The revised chapeau would read as follows: "Without hampering progress in or access to peaceful uses of intelligent autonomous technologies, prior to potential improvement and as appropriate throughout the entire cycle of laws, States should." Second, as suggested by several States, including Australia and the Russian Federation, this section would benefit from streamlining, including merging and deleting certain paragraphs. Additionally, while we will not reiterate our position on the term "context-appropriate human control and judgment" in paragraph 2, we maintain that a more appropriate formulation should replace the term or remove the paragraph, as suggested by others. Lastly, regarding the phrase "harmful bias" in paragraph 6 and 7, we share the concerns raised by our Australian colleague. The term "harmful bias" is subjective and raises the question, "Harmful to whom or to what?" But we also understand that some delegations find the term "unintended bias" we used before uncomfortable. So we propose to replace it with "bias in data sets or other algorithmic biases" or data, algorithmic, and other biases. Thank you.

Let me thank the distinguished representative of the Republic of Korea. Next on my list is Denmark, to be followed by Colombia and Mozambique. But let me now first pass the floor to the distinguished representative of Denmark. You have the

Thank you, Mr. Chair. My delegation is pleased with this section of the rolling text, and we can accept it as it is. We wanted to highlight that we appreciate the inclusion of paragraph 2, which talks about ensuring that laws are operated with context-appropriate human control and judgment across the entire lifecycle of the weapon system. Furthermore, we wish to emphasize the importance of conducting legal reviews of weapons, means, and methods of warfare, and the implementation of risk mitigation measures to ensure compliance with international law, in particular IHL, and to address relevant ethical concerns of laws. We would also like to add our voice of support to the elements of this box addressing the question of bias. Risk mitigation measures should address the harmful effects of potential biases instilled in the weapons system through the application of artificial intelligence, including biases related to gender, ethnicity, age, and disability. Thank you, Mr. Chair.

Let me thank the distinguished representative from Denmark for sharing her views and making suggestions. Next on my list is Colombia, to be followed by Mozambique, India, Italy, and Norway. But first, now the floor to the distinguished representative of Colombia. Madam, you have the floor.

Thank you, Chairman. In general, like others have said, our delegation is in agreement with much of the elements here in box four. We'll just make a couple of additional remarks. First, the first measure that's very important for us, it's important to keep paras two and four. On para two, we reiterate an earlier comment that we made With respect to box 1, instead of "context-appropriate", we would prefer the word "meaningful" human control and judgment. As per what we proposed in protocol 6, on para 4, hearing from other delegations, what they said, For us, it is important to keep it, and it is in line with what we proposed in protocol 6 on the theme of review. Finally, on paras 6 and 7, on the theme of bias, in view of the proposal for protocol 6, We support what was said by the ICRC, especially it would be important to distinguish between automated bias from algorithmic bias. We used to draw a distinction there, and perhaps there we could have two paragraphs in that respect. In Protocol VI, there's a reference to particular I won't read it out here, but it goes very much in line with what the ICRC mentioned. And obviously we are open to the best way of including that, those references, especially when it comes to the theme, when it comes to including the word harmful, or would it be better to put instead of harmful, unintended? Thank you.

Let me thank the distinguished representative of Colombia for her comments and suggestions. Next on my list is Mozambique, to be followed by India. But let me first pass the floor now to the distinguished representative of Mozambique. You have the floor.

Thank you, Mr. Chair. As it's Mozambique first time to take the floor in this mandate, allow me to congratulate you and your team on this work done during this presidency. We should also like to thank the friends of the Chair for their work on the operationalization on this rolling test. On box four, we understand this box addressing the issue of weapon review. We agree with this box at this is, but we should like to suggest In addition, for Mozambique, cybersecurity concerns are part of the issue that should be addressed during the legal review. We already have agreed language on cyber concerns on the 2019 GGA report, namely in the guiding principle F, which said that during the development or acquisition of such system, physical security appropriate and non-physical safeguard, including cybersecurity against hacking or data spoofing. The risk of the acquisition by terrorist group and the risk of proliferation will be considered. We believe this consideration should be included in the legal review of such system and therefore will fit in the box four. Thank you, Mr. Chair.

Let me thank the distinguished representative of Mozambique for sharing the views of his delegation and making proposals. Next on my list is India to be followed by Italy, Norway, and Austria, but let me first pass the floor to the distinguished representative of India. So you have the floor.

Thank you, Mr. Chairperson. Let me at the outset begin by appreciating you for steering the work of the group. Your job has, of course, been difficult given the divergences of views that we have seen, but we are sure that you will be equal to the task that is entrusted upon you. Before I address the section four, we want to acknowledge the in-depth discussions that have taken place this week on various sections of the rolling text, which I must say is very encouraging. We have seen some very active participation and very high quality of argumentation, which reflects that the group remains the authoritative forum for discussing lethal autonomous weapon systems and autonomy in weapons, and that it has the necessary expertise to grapple with these issues. On section four, we view this as an important section. There are several reasons to this, but the most important is that the section could potentially become a placeholder for those elements of the rolling text Which have failed, uh, to find convergence in, uh, the sections above, particularly those on regulations and prohibitions. I have four points to, uh, make. The first is on the chapeau of this section. We would like, uh, we would prefer to use, uh, uh, the word states are encouraged to, instead of the formulation now where, uh, we have stated states should. And with the amendment, the shape would look as follows. Prior to the potential employment and as applicable throughout the entire lifecycle of laws, states are encouraged to. We want to make a distinction between the regulation measures that are covered under Section 3 and those measures which are in the nature of risk mitigation. And there is an intrinsic difference between the two, and we view the risk mitigation measures as adjunct to the measures that are that are primary obligations for the purpose of the rolling text, and our amendment seeks to clarify that relationship. We can also support the suggestion made by the Republic of Korea that these measures in and of itself should not hamper the progress of SND advancements which are relevant to the Convention. Second, in paragraph 4, we believe that this paragraph is, uh, is, uh, it has, has a great overlap with what is contained in paragraph one. Uh, we would also therefore, uh, uh, request if the same qualifi, qualifiers that are built in paragraph one are also built in for paragraph four, which is that these are, are to be in accordance with the obligations, state obligations under the international mandatory law. We are also open to deleting this paragraph because we believe that this provision is by and large captured in the paragraph one of the rolling text of this section. Third, with regard to paragraph five, we see that there is a reference made to conducting rigorous testing and evaluation in realistic simulated operational environments. We are open to a reference to this. But the following part of the text which conveys that it enables a human user to have a reliable expectation, the terms such as reliable expectation and anticipated circumstances of use have not been used elsewhere in the text. So we would request if these be dropped and it does not really compromise on the scope of the measure that is given in paragraph 5. Fourthly, we feel that there have been, there has been some divergence, uh, and opposing views on some sections, uh, on some parts of the, uh, rolling text in sections above section 3, uh, such as concepts of traceability, explainability, predictability, and, uh, uh, transparency, and a view was expressed today that these are to be, uh, are to be viewed, uh, uh, these are broad labels and they are to be seen as consistent with the, uh, uh, consistent with the meaning that countries would derive in their national implementation from the international humanitarian law. So it is our view that given that we have divergences on this, on, in section three, this could be, uh, uh, this could be a placeholder where some of those terms could be referred to in a manner that states are encouraged to take them into account instead of creating mandatory obligations, consistent with the revisions that I have suggested in the chapeau of this section. Last but not the least, we also agree with views that the duplication of concepts that we see in other parts of the section should not, that redundancy should be done away with or that duplication should be dealt with. We can also agree with the views expressed by the delegation of Australia on, uh, their amendments on paragraph six and seven concerning, uh, references to harmful, uh, bias and the, and, and the amendments they have suggested on that. I thank you.

Let me thank the distinguished representative of India for his, um, reactions to the text, uh, and making proposals. Next on my list is Italy to be followed by Norway, Austria, Spain, but let me first pass the floor to the distinguished representative of Italy. Madam, you have the floor.

Thank you, Chair, for the work you have done on this section of the rolling text. Italy is of the view that this section offers an opportunity to strengthen the responsibility of states in relation to the potential use of LOAs. For this reason, although we are aware of divergences in this conference room, we would like to reiterate that we would favor the substitution of "should" with "must," as highlighted a few moments ago by the ICRC, also noting that "should" would appear in contrast with the rigorous testing that we find later in paragraph 5. We discussed a lot yesterday the relevance of development, and even in relation to this section, we believe that laws may be developed if their effects are able to be anticipated or controlled as required in the circumstances of their use. In addition, the activity of testing described in paragraph 5, in our view, does not need a specific paragraph. and it could be better reflected as an additional sentence to paragraph three that would read as follows, "To this end, the state must conduct rigorous testing and evaluation in realistic simulated operational environments." For us, this suggestion would help streamlining the text without detailing the activities involved. In addition, we feel that there is an important element that is missed and that has already been stressed, and which pertains to appropriate technical and physical safeguards to the risk of low proliferation and acquisition by non-state actors. For this reason, we would like to make the following suggestion of an additional paragraph that would read as follows. States must ensure technical and physical safeguards during the design, development, modification, and testing of laws, including non-physical safeguards, cybersecurity against hacking or data spoofing, and appropriate safeguards to the risk of proliferation and acquisition or use by non-state actors. And this, in our opinion, would be in line with the guiding principle F. Finally, following up with the discussion we had yesterday, we would like to express support to the suggestion of the distinguished UK delegation to move para 6A of section 3 to this section as it would better fit the related technical content. And I thank you, Chair.

Let me thank the distinguished representative of Italy. for her comments and suggestions made. Next on my list is Norway, to be followed by Austria, Spain, Belgium, France, Ireland and USA. But the floor goes first to the distinguished representative of Norway. We have the floor.

Thank you, Chair. Box 4 includes a range of important measures, we think, to reduce risks and promote compliance with the law. On a general note, the text could probably benefit from a bit of streamlining, for instance, by co-locating or merging related issues, paragraphs 3 and 5, for instance, and 1 and 4, also to help avoid that apparent redundancy. And generally speaking, we support the elements that are in there. One concern we have with the text is the relation between the chapeau and paragraph 1, like others have mentioned. a bit earlier, that the first sentence of paragraph one calls for the conduct of legal reviews, which is a legal obligation pursuant to Article 36, uh, which does not square with the word, use of the word should in the chapeau. Uh, as, as we understand it, the purpose of paragraph one, apart from reiterating a legal obligation, is to promote the voluntary exchange of best practice, which we agree with. So one fix could of course be to write must in the chapeau, but that could create problems of its own since the chapeau of course prefaces all the paragraphs in this box. So we propose to fix at least this particular problem by combining the two sentences in paragraph one and changing the order. So something like this, and I can send that to you by e-mail, engage as appropriate in the voluntary exchange of relevant best practices between states. in order to facilitate the determination in accordance with their obligations under international law in the study, development, acquisition, or adoption of a new weapon, means, or methods of warfare, whether its employment would, in some or all circumstances, be prohibited by international law. That way, it's clearer that this should only relate to the exchange of best practices. Regarding paragraph two, we think first that it could be useful maybe to unpack the notion of the life cycle and spell it out. So design, development, acquisition, and so forth. And second, with the current formulation, I mean, it seems to be mixing up the timelines a little bit. As we understanding, operating a weapon system is something you do towards the end of its life cycle. You can't operate a weapon throughout its life cycle as the sentence now reads. So The notion that human control is something that should be ensured throughout the various stages is something that we support, but we think that the paragraph needs to be reformulated to reflect that better and for reasons of internal logic in the paragraph. On the issue of bias, we agree that the two kinds of bias that are mentioned in paragraph 6 should be split into separate paragraphs, since they are very dissimilar issues. Data bias is a phenomenon that pertains chiefly to the development phase, whereas automation bias is a human control issue. Thank you.

Let me thank the distinguished representative of Norway for his comments and the suggestions made. Next on my list is Austria, to be followed by Spain. But let me first pass the floor to the distinguished representative of Austria. Sir, you have the floor.

Thank you very much. Um, this is also a very important section because it, it gives some additional indication on what states, um, need to, uh, but also should do. Um, maybe this is a little bit of a problem that it mixes those two elements and some have already pointed to that it repeats, uh, obligations that exist and that are very strong for, for many of us or for, uh, most of us, especially when it comes, to point one. So there is definitely a need to change the chapeau and to take out the should and maybe, I mean, one easy solution is to just have the different verbs in front of every section so that we, for example, would use the first one, shall determine in accordance with their obligations and then just continue with the formulation from Article 36. Um, this is a important point and many others have made it, uh, how, how we deal with the second, uh, sentence, if, if we separate it, if we took this element out, uh, or if we keep it like this, I mean, there, there we are flexible, but there is, is some, some work to be done here and, and we are open for, uh, suggestions. Um, then in, in point, para two, uh, I think the main idea behind this para is to have the life cycle of the weapon systems as something that states have to keep in mind and something that states need to ensure. Even here, we might see a possibility to use the word shell, as it is a very important para and something that needs to be maintained. But then it's very much linked to the previous box. The question that other colleagues have brought up, how is it related to box three and box four? I mean, one solution would be to have a strong formulation in box three and here spell out what the life cycle of the weapon systems means. And suggestions by Norway have been very useful in this regard. But there needs to be some differentiation on the purposes of the two elements. But as I said at the beginning, the life cycle element is something that is important and the idea here, as I understood your suggestion, is that there should be some element of constant care by states when it comes to this element. Then, under para 3, we support the idea of deleting properly by the Russian Federation. Um, we would even add, uh, at the beginning, and this is a formulation I think that is established that we use in its normal or expects, expected circumstances of use, um, as, as this makes, um, it more, more direct and also more clear, uh, what kind of, of circumstances, uh, we are thinking about in para four, um, we also… think that this is an important part that we should probably maintain. What could be a useful change is to kind of spell out what the, someone called it the operation, but we could also use the words functions or effects of the existing weapon systems would be so that this is more specific about what is to be what is to be -- what would be the change that would occur, so that we don't have to speculate what this could be. Again, in para 5, we also would support that we used any normal or expected circumstances of its use in line with the expected circumstances that we have in para 3, so that there is that we are using the same kind of formulations. Now we are coming to the issue of bias, which is, I think, a very interesting and important element, and there has been some deep discussions about this, and we feel very strongly about how this has to be indicated and used in this context. I think there is a general problem that we are having already a large set of discussion on algorithmic bias, on other type of biases that relate to new technology, on bias and data sets in the civilian sphere. And some colleagues, I think they try to indicate that there is a specificity to the military sphere when it comes to the use of algorithms. I think this has to be looked into and also tested because in general, the use or the build up of algorithmic systems would probably be the same. I would, the main difference is probably that military systems will be under more care and scrutiny as the effects are more dramatic and probably more risky. But the general problems that we are seeing are probably the same, but we are here on something that we still have to see how it how the evolution of these systems will happen. One point that has been brought up by colleagues is that there could be unintended bias. This is a very interesting question because it also indicates that there could be something as intended bias. A question that we have brought up already in the past is that intended bias is very difficult to differentiate it from normal training of an algorithmic system. In the civilian discussion, there is also the issue that bias does not relate to kind of something that could be discrimination or differentiation. It relates, algorithmic bias relates to a whole host of different problems that goes from confirmation bias to feedback loops to just dirty data sets and other elements. So we have to look into exact wording here, because the concept that we are using here or thinking here, which means there is a system, it differentiates, it discriminates, is very narrow. The algorithmic bias discussion is much broader and includes many other problems. And the common denominator is that there is an error in your system. military users probably don't want to have an error in their system because it makes their system unpredictable and also less useful in a military context. So the question is, is in the military field really a scope or purpose for intended bias if it's considered as an error? But we have to discuss what we are actually discussing here about. Just to make this point. On general, on para 6 and 7, some colleagues have already pointed to it, there's probably we should separate it or rearrange those two paras. From our point of view, there are not only two different topics here, but three. First, there is the issue of data sets. Data sets also refers to quality and integrity of data and other problems and issues that are related to data sets that not per se relate to bias. So probably we should have an own para on data and how it is used. and, uh, that, uh, care that states should, uh, take when selecting data sets and using them, then there should be a para on the issue of bias and open to formulations and, and, and suggestions from states in order to, uh, to, to, to, to, to, to, to, to, address the questions that we have just raised. And then there should be a separate bar on automation bias, because this is a different phenomenon. And we would argue that we should not mix it up with algorithmic bias or data bias and data sets, because automation bias relates much more to the issue of nominal human input. lack of training, degeneration of, of, um, skills in humans and their constant care. So there is something that we have to, to separate here, and we would suggest to have, uh, three paras on these issues. I thank you.

Let me thank the distinguished representative of Austria for his comments and the suggestions made by him. Next on my list is Spain to be followed by Belgium and France, but let me first pass the floor to the distinguished representative of Spain. Madam, you have the floor.

Thank you very much, Chair. Thank you for giving us the floor once again. And I'd like to begin by conveying to you our congratulations on the very able way you're steering this very intense set of deliberations. Now, on box four, that we're dealing with currently, which is one of the critical parts of this document, we have the following comments to make. The proposal is one that we mostly agree with. It makes a major contribution. Responding to points made by various delegations with regard to the chapeau, we believe it's important to keep the word "should" we would have reservations in expanding the language beyond the use of the word "should". On paragraph 1, we agree with it because it compiles agreed language from Protocol 1 of the Geneva Conventions. We also agree with the wording and drafting of paragraphs 2 and 3, which was done on the basis of guiding principle C, which provides for the human-machine interaction, and that is a requirement throughout the entire life cycle of the weapon systems. With regard to paragraph 4, we have some reservations about the use of the word "significantly", given that there can be various different interpretations of what exactly is meant by "significantly", and it can have a different degree of application or weight. On paragraph 5, we agree that with, because again, it is aligned with the guiding principles, this time guiding principle G. In paragraph 6, in order to contribute to consensus, we can adopt and accept it, but we think that the following wording could make the point broader and would provide with a better link between paragraphs 5 and 6 and 7, and I will read our proposed wording in English, ensure the critical safety features have not been degraded.

Processes such as monitoring.

Finally, we agree with the wording proposed in paragraphs 7 and 8. Thank you very much.

Let me thank the distinguished representative of Spain for sharing her views. and comments. Next on my list is Belgium, to be followed by France and Ireland. But let me first pass the floor to the distinguished representative of Belgium. So you have the floor.

Thank you, Mr. Chair. Box 4 looks very good, thanks to your efforts. Nevertheless, we have three remarks. First, in para 1, We think it would make the message stronger and also more readable to start para one by the sentence, "Conduct legal reviews to determine in accordance with their obligations." So to add the terminology of legal review already in the first part of the para. Secondly, With regard to para 4, so there is, there, this is about weapon systems under development or modification that significantly changes the use. We think it's better to reformulate in the following sense. So we would like to change the para as follows. Ensure that the legal review is renewed if there are significant modifications to the weapons core functions and actions. This is in our view, a clear, this is also a sentence we sent to you a few weeks ago. And then thirdly, this is about the issue, the risk of diversion of laws to unauthorized end users such as terrorist organizations or organized crime, which is also referred to in guiding principle F. So we would like to and and there have been some suggestions in this field, we would like to propose the following formulation: States should ensure the adoption of measures to prevent, stop and address the diversion of laws to unauthorized end users. We also would like to use the word diversion because this is a word that is used in arms control, in strategic export control, so we think that there is good reason to use the word diversion also in this new paragraph. Thank you.

Let me thank the distinguished representative of Belgium for sharing his views and presenting proposals. Next on my list is France, to be followed by Ireland and the US. But let me first pass the floor to the distinguished representative of France. Madam, you have the floor.

Thank you, Chair. On box 4, We support the language currently to be found in the chapter, and we also fully support the current drafting of paragraphs one and two. On paragraph three, we would like to amend the word properly and replace it with the word sufficiently, which is language that we use on a standard basis to reflect the due flexibility that is left to the command chain. We want to keep paragraph four and the word significantly. because that justifies a review of certain systems, including the review of their lawfulness. In order to determine whether a weapon is new, My apologies, I just need to refind my place. In order to determine whether a weapon is a new one and should be subject to review, we need to determine whether it is distinctive in terms of its effects from other weapons that exist in the arsenal or use doctrine currently in force. We would like to know whether this paragraph would cover both modifications to use doctrines but also significant modifications to the effect of the weapon itself. Would both of these concepts be covered by paragraph 4 as it's currently worded? On paragraph 5, just like India, we don't like the wording "reliable expectation." We'd like to replace that wording by the wording "appropriate knowledge," which seems to us to be language that is more commonly used in IHL. As explained by Australia, and the point was echoed by other delegations, in paragraph 6, we support the deletion of the word "harmful", and we'd make the same point in paragraph 7, where we'd also delete the word "harmful". Currently, we would be in favour of the current suggestions with regard to test and evaluation, how these could be included in paragraph 6 and 7. We support paragraph 8, that's vital to ensure that there is human control. We could also insert a paragraph on technological progress along the lines suggested by the Republic of Korea and duly seconded by India. Thank you very much.

Let me thank the distinguished representative of France for her comments. Next on my list is Ireland, to be followed by the US, the UK, and Israel. But let me first pass the floor to Ireland. You have the floor, sir.

Thank you, Chair. And as it is my first time to speak, I would like to echo the positive remarks of my Indian and Spanish colleagues on your effective chairing. Ireland agrees with the ICRC and our Italian and other colleagues on changing "should" to "must" in the chapeau as one option to meet obligations. But we are open to considering constructions proposed by our Norwegian and Austrian colleagues. Notwithstanding that, we also agree with our Canadian colleagues on switching employment to deployment for consistency. On para 3, we agree with previous recommendation from the Russian Federation to remove properly before understood, as this is a subjective term. And while listening to our French colleagues' proposal, we believe understood, full stop, is more clear. On para 4, we support the ICRC suggestion, including function and possible effects alongside use. We also wish to see the inclusion of acquisition alongside development and modification, which is in line with language earlier in the section, for example, in para 1. We welcome paras 6 and 7 regarding data bias and automation bias. We think these can be fleshed out a bit and rearranged to appropriately address the two separate concepts as a part A and B of para 6. Therefore, we recommend deleting para 7 and inserting "take measures" into para 6 and after "detect and." We share the view of Australia on para 6 that periodic assessments could be understood in a variety of ways, and we appreciate the proposal. We recommend adding to a sense of frequency, though, with which assessments on data sets will be needed. by including the phrase, "regular and periodic assessments at appropriate points through its life cycle." The basic principle is that this should be an ongoing exercise. We further support deleting as much as possible, which we see as an unnecessary caveat. Finally, we support the ICRC suggestion to separate data bias and human cognitive tendencies, such as automation bias, and this would be into the parts A and B. And we agree with the general point of our Austrian colleagues that we need to look deeper into discussions on bias. Thank you, Chair.

Let me thank the distinguished representative of Ireland for his, his comments and the suggestions made, uh, also referring to other proposals already made. Um, next on my list is the US, so you have the floor.

Giving our delegation an opportunity to provide comments on this box. Recognizing your approach to titles at this stage, I'll just state my delegation's understanding here that we're talking about measures. And really, I mainly make that point just to give a sense to colleagues about the rationale for our proposed revisions to these specific paragraphs We're trying to make these things implementable and concrete and effective. And so that's the spirit with which we're proposing a number of revisions to this section. With regard to the chapeau, the comment we would make just relates to the phrase "as applicable throughout the entire lifecycle of laws." In our understanding, some of the measures occur in specific parts of the lifecycle and not others. Therefore, we would propose to revise the chapeau to read, "prior to potential employment and when relevant in the life cycle of loss." Before getting to paragraph 1, we wanted to propose another measure, potential measure for the group to consider. We were inspired by the intervention we heard from our colleague from Bangladesh yesterday, who noted the importance of design as an area that we need to be thinking about in this context. Therefore, we would propose a specific measure related to design, and that would read as follows. Refrain from designing laws to target civilians or civilian objects or to spread terror among the civilian population. So we see that as a specific measure that's very concrete and implementable, and we're hopeful colleagues can support this proposal. With regard to paragraph one, I think our suggestion would be to replace new weapon, means, or method of warfare with laws. And I think the proposal here is just in the spirit of our mandate, we're not engaged in an exercise of abstractly repeating various rules of IHL. We're trying to develop a set of elements of an instrument and other possible measures to address emerging technologies in the area of laws. With regard to paragraph 2, I think we could support Japan and Canada and others' proposals to remove this paragraph. I think we would endorse the observation that Norway made, which we thought was very insightful about operation being only one part of the life cycle. So to the extent this language is being considered in other contexts, I think it's very useful to bear that comment from our Norwegian colleagues in mind. you know, recognizing our positions just with regard to the phrase context appropriate human control and judgment. We did want to offer, try and be as constructive as possible and offer an alternative, you know, that could be hopefully useful to consider. And this is inspired from paragraph 23A of the 2019 report. which lists a number of risks and in particular includes discussion of the risk of unintended engagements. So our proposal for paragraph two would be to say, take appropriate measures to mitigate the risk of unintended engagements in the use of laws. And so for us, this is a core objective and principle in the responsible use of autonomy weapon systems. We want to ensure that the use of these weapons reflects the human agency and intention, the intention of the commanders and operators to strike military objectives and to avoid harm to unintended engagements, like in the case of civilians, for example. With regard to paragraph three, I think we think there are some good ideas in this paragraph, but we would encourage perhaps moving it either below in the section or moving it to the next section. The key point, I think, that we think might be useful to clarify and address is who is doing this understanding, and then in particular to address what specific steps are being taken to ensure understanding. And so we see this concept as very pertinent to discussion on training, on discussion on guidance. So I think our principal recommendation is to take some of this language and either move it below or move it to the next section. So it really gives a purpose to some of the measures that we're offering. With regard to paragraph four, first of all, I'll just read out our proposed revision. So we would propose that it read, "Review as applicable laws under modification that significantly changes its use to ensure it can be used in compliance with IHL. And there's some edits here for clarity. There's some edits to just focus on laws because that's the main issue that we're focused on here. I think we also took on board the comments that development, legal reviews and development are already addressed in paragraph one. So it could be useful to have an additional paragraph really just focused on modification With regard to paragraph five, I'll read out our proposed revision, you know, for clarity. Conduct rigorous testing and evaluation, including realistic simulations of its use in operational environments. to enable a reliable expectation of how the laws will perform in the anticipated circumstances of its use. So we've proposed a change with regard to the language about realistic simulations, and the point we're trying to make is that the testing and evaluation doesn't necessarily occur in the operational environment, but it can occur in other contexts, but I think what's intended is really this idea of simulations of its use in operational environments. So we're really just trying to clarify that concept. We do think the phrase "a human user to have" can be omitted, simply because the direct products of the testing and evaluation are reviewed by a number of officials, and it's often not directly the users who are reviewing these materials. And then, as above, really focusing on laws as opposed to just any weapon system, bearing in mind the mandate of the group. With regard to paragraph 6, there's been a lot of really good discussion about this paragraph and the next paragraph. I think our specific proposal, you know, would be to omit the first clause, conduct periodic assessments to detect and correct as much as possible harmful bias in data sets. And our thinking there is we see data bias as being addressed in paragraph seven and already being covered there. You know, in our view, maybe Periodic assessments is overly specific, and we prefer the broader formulation that's used in paragraph seven. And then I think ICRC, Norway, Austria have made very pertinent comments about trying to separate the issues of automation bias from the issues of AI bias. to me or to our delegation, it makes a lot of sense to really put these in separate paragraphs because they are distinct issues. So I'll read out our proposal with regard to the second-half of paragraph six. So we propose at read, implement measures to mitigate the risk of automation bias in the use of laws that may increase the risk of the loss of civilian life, injury to civilians, or damage to civilian objects. So really the intent of this last clause is really to emphasize the context of the CCW and our overarching goal of enhancing the protection of civilians and complying with IHL. We also think it's useful to add the risk of before automation bias. With regard to paragraph 7, we did want to endorse the proposal by Australia, ICRC, I think, which others have supported, to replace "harmful" with "unintended." We would have two other proposals with regard to paragraph 7. One is to change "relied upon by" to "integrated into." And we think this would avoid anthropomorphization. And then similar to above, we think it would be useful to really relate this idea of bias to the protection of civilians. And I think our concern is, as our colleague from Austria was explaining, bias can relate to many different attributes. It could relate to military effectiveness, it could relate to many other concerns, considerations that are not necessarily specific to the things that we're focused on about in the CCW. So relating it to the protection of civilians, I think, would be quite useful in focusing its implementation and focusing on the mandate of this body. So I'll just read out our proposal with regard to paragraph seven. Implement measures to address possible unintended bias in artificial intelligence capabilities integrated into laws that may increase the risk of the loss of civilian life, injury to civilians, or damage to civilian objects. And then with regard to paragraph eight, I think our first preference would be to delete this paragraph and instead address its substantive provisions in the next section in a more detailed way. So it's really an intent to expand upon what we think is a very important measure here. I'll note just a few technical comments with regard to the language here. I think we would see instructions as somewhat redundant with guidance and training. Training could involve instructions, and then instructions are sometimes used as an example of guidance. So our recommendation would be to omit instructions and instead refer to guidance and training. You know, we also recommend replacing human with their. And I think this would clarify that states are performing these activities in regard to their users of laws rather than users of laws by other states or actors. And then here could also be an opportunity to draw on the useful language which was provided above about proper understanding of the capabilities and limitations of laws and the expected circumstances of use. So I'll read out one option. I think that we could support with regard to this paragraph, states should provide appropriate guidance and training for their users of A-Laws in order to provide them with a proper understanding of the capabilities and limitations of the laws in the expected circumstances of their use. I thank you, Mr. Chair.

Let me thank the distinguished representative of the USA for his views and comments and extensive proposals for a review of the text. Thank you in advance, or we might have it already, for sending them in writing, because this was quite detailed. Next on my list is the UK, to be followed by Israel, and we have now also Peru, Cuba, Switzerland, and Brazil. But first, I would like to pass the floor to the distinguished representative of the UK. You have the floor.

Thank you, Mr. Chair, and thank you for giving us this opportunity to comment on box four. So moving on straight in, with regard to paragraph two, the UK thinks that in order to avoid mixing terminology here, we suggest avoiding describing laws as being operated across the life cycle. Therefore, the UK will suggest the following small change to this paragraph, which would read: "Ensure that context-appropriate human control and judgment is applied across the law's life cycle." With regard to paragraph 5 and the concept of reliability of performance, In the UK, we fulfil the need to understand the capabilities and limitations of laws by understanding their performance across different situations. Therefore, the UK suggests the following amendment to paragraph 5, which would read: "Ensure that the performance and effects of laws in expected circumstances of use are understood providing a clear understanding of its capabilities and limitations in relevant operational contexts. Regarding paragraph 6 and the mitigation of automation bias and addressing comments made by a number of delegations already, and particularly expanding on the intervention made by Norway, I think we do agree that it is worth clarifying what is meant by automation bias. In the UK, we regard automation bias as the propensity for humans to favour suggestions from automated decision-making systems and to ignore contradictory information made without automation, even if it is correct. So we think that this concept fits better in paragraph eight rather than in paragraph six. This is because paragraph six, we think, focuses on technology rather than the human element. This change would also highlight the important role that the training of human users and those in the responsible chain of command would play in human understanding of system behavior, performance, and the limitations across operating conditions in order to support context appropriate human control and judgment. So therefore, with references to both paragraphs six and eight, the UK suggests the following. So with paragraph six, we would suggest that that would read, conduct periodic assessments to detect and correct as much as possible unintended bias in data sets and AI models. and delete the rest of the paragraph. I note that our proposed amendment supports the original suggestion by Australia and one that has been supported by many delegations already, to amend "harmful bias" to "unintended bias". That change would apply also in other areas of the text where the word "harmful" occurs. Regarding paragraph 8, we would suggest it would read, "States should ensure appropriate guidance, training, and instructions for human users of laws as a means of enabling context-appropriate human control and judgment and mitigating the risk of automation bias." Alternatively, we could support splitting paragraph 6 into two, as other delegations have suggested, in order to affect the point that we have made in the need to clearly split out bias as it relates to technology and as it relates to human bias. Just moving back quickly to paragraph 6, the UK also suggests that we clarify the wording around periodic assessments by specifying that there needs to be both evaluation of bias during the design as well as ongoing monitoring of bias. Thank you, Mr. Chair.

Let me thank the distinguished representative of the UK for his comments and the suggestions made with regard to the text in front of us. Next on my list is Israel, to be followed by Peru and Cuba. But let me first pass the floor to the distinguished representative of Israel. So you have the floor.

Thank you, Chair. We will present our comments to section four. Regarding the second paragraph, First, with respect to the term content appropriate human control and judgment, we will refer to our previous comments. This term is highly controversial. It may be counterproductive in promoting a common understanding regarding the appropriate human machine interaction. Second, this provision seems redundant as the issue is already addressed in section 3. In this regard, also note that the chapeau of this section concerns pre-employment measures, while this provision addresses the operation phase. For these reasons, we suggest, as our US colleagues, deleting it or alternatively deleting it from paragraph 6B in section 3 and addressing it only here while adapting its framing to pre-employment measures. Regarding the fourth paragraph, Similar to the position presented by the Indian delegation, we find that the relationship between this paragraph and paragraph 1 under this section, which also deals with weapons reviews, is not fully clear. Also, the caveat in paragraph 1, I quote, "in accordance with their obligations", is equally relevant here as well at the end of this paragraph. Regarding the fifth paragraph, We suggest using the word appropriate instead of rigorous to emphasize the need for testing that is suitable and aligned with the context and specific operational requirements of the weapon system. Regarding the sixth paragraph. The issue of bias in AI and, relatively, datasets is addressed in the next paragraph, so we think that the current text is redundant. The next paragraph also addresses it better because it does not prescribe the exact measures to be taken and allows states to consider the best possible way in each case. We will suggest, therefore, leaving only the issue of automation bias in the current paragraph, which is a different thing than data-related AI bias. For these reasons, we suggest that the paragraph contain only the following sentence: "Implementing measures to mitigate automation bias." Finally, regarding the seventh paragraph, Bias is a subjective and judgmental term. In a sense, in AI, there is always bias towards the data set that was fed to the system. And the real question is whether the bias was deliberate and legitimate, such, for example, as bias against enemy tanks or not. For this reason, like other delegations such as the US, Australia, India, and Canada, We think that the word "unintended" should be added here. Otherwise, the qualifier "harmful" may be insufficient because it's too vague. Thank you, Chair.

Let me thank the distinguished representative of Israel for sharing his comments and making proposals. Next on my list is Peru, to be followed by Cuba and Switzerland. But let me first pass the floor to the distinguished representative of Peru. Sir, you have the floor.

Thank you, Chair. Thank you for the new version of box 4. I think it's a step in the right direction. Regarding the chapeau, we would like to support those who think that the best verb to use should be "shall" or "must". as this is what Italy, Iceland and the ICRC said among others. This is what Norway said. This is explained because in para 1, there's a reference to obligations under international law and that obviously would not be consistent with the chapeau that would just say "should", because these are international obligations. Still in para 1, we support the proposal to change the words "best practices" with "relevant" or "pertinent practices". Para 2, we would add at the end of the life cycle of the weapon system, we would add wording, a sentence that's contained in the Protocol 6 proposal from the G16 that refers to, I'll read it in English.

The self-learning capabilities of systems.

On 3 and 5, we would support together with Austria and the ICRC, adding the word, the expression, "normal or expected circumstances". And we'd also delete the word "properly" before the word "understood". On 6 and 7, the discussion on bias. We support those who suggested the need to distinguish clearly between the automation bias from harmful bias. And finally, we noted with interest the proposal of a new para 9 as raised by Belgium. to do with adopting measures to prevent and stop diversion of automated weapons to non-authorized users. So the scope, we would suggest, the scope of that proposal should be adequately considered by the group. Thank you.

Let me thank the distinguished representative of Peru for sharing the views of his delegation and making proposals. Next on my list is Cuba, to be followed by Switzerland. Let me first pass the floor to the distinguished representative of Cuba. Madam, you have the floor.

Thank you, Chairman. As we've said previously, both in this session, both this section and section 3 are very important for us. but it's not enough if we're seeking to achieve a more ambitious goal that could lead to a legally binding instrument. Because the proposals in this session are geared towards mechanisms similar to, for example, TCBMs, which kind of takes us away from the creation of an actual LBI. Therefore, we share the views of those countries, have noted that the verb should be changed from should to something like shall or must, so to be more constructive and to keep, in order to have a consensual document. Moreover, Cuba would also like to see progress in technological progress in the civilian sphere, which is outside the realm of the group of experts. We agree that human control throughout the life cycle, above all in the concepts linked to international humanitarian law, is a responsibility. Therefore, we do not agree with deleting paragraph 2. We would be against that. The idea of leaving In paragraph 8, for human use, it's important to keep the element of human control. And we agree with Peru and Belgium to create a new paragraph, para 9, on diversion of laws to non-authorized users. I think that would be prudent given the debates that we're having. I think we also need to dig deeper into the issue of bias in programming. I don't know if it's feasible to separate the two kinds of bias, as is set forth in paragraph 6.

Let me thank the distinguished representative of Cuba for sharing her comments and making proposals. Next on my list is Switzerland, to be followed by Brazil, and then I have Article 36 and Bulgaria. But first, the distinguished representative of Switzerland, you have the floor.

Thank you, Mr. Chair. Allow me to start with a comment on the chapeau. As it stands now, it would relieve non-state parties to an armed conflict of their responsibilities. While it is primarily states that have the means to implement the following measures, it would be wise to consider whether non-state actors, in particular parties to the conflict, should also be held accountable. Moreover, We also believe that the should should be strengthened as the following paragraphs either already reflect existing obligations or should be clear commitments in a new instrument. Moving on to paragraph one, we believe that in a new instrument, especially if it was a legally binding one, states should clearly commit to conduct legal reviews of lethal autonomous weapons systems instead of simply restating existing law. After all, states have an obligation to respect and ensure respect for IHL and thus need to ensure that the weapons they intend to use can be used in compliance with IHL. On paragraph 4, This seems closely related to the obligation to conduct legal reviews, and we would either combine or at least put it together with paragraph 1, especially the phrase under development falls in our view already within the legal review, which is dealt with in paragraph 1. We moreover would support the proposal of the Asian Pacific Institute for Law and Security to include the word deperformance or use. And moreover, we are of the view that in this case, the word significantly could actually be deleted as any modification that affects the performance of a system must undergo a form of a review that is adequate for the changes or modifications made. Moving to paragraph 6, a number of points have been raised on bias. Allow me to make two points here from our side. Like others, we recommend revising paragraph six and seven to emphasize that harmful bias in AI capabilities relied upon by laws and automation bias are two distinct risk factors. This distinction ensures better clarity and a more focused approach to address the risks associated with laws and avoids duplication. We also think it is advisable to focus on unintended bias. A system designed with an intentional bias against targeting civilians or civilian objects can reinforce IHL principles, particularly distinction and proportionality. A well-calibrated bias can also be introduced to favor de-escalation. While Unintended or unaccounted for biases can be problematic in laws. Deliberately designed biases can be used to strengthen adherence to IHL. So in our view, the para could focus on unintended harmful bias. Lastly, a comment on paragraph 8. In our view, this should, paragraph 8 should not be a guidance or recommendation as it is now with the word should, but we believe this would merit to be a clear commitment and even reflects existing legal obligations more generally with regard to IHL. For instance, Article 80 of the Additional Protocol I requires that high contracting parties or parties to a conflict shall give orders and instructions to ensure observance of the Geneva Convention and its additional protocols and shall supervise their execution. The idea of dissemination and instructions is not new and, for instance, is also part of the CCW in Article 6 and its protocols, for instance, Amended Protocol II, Article 14. I thank you, Mr. Chair.

Let me thank the distinguished representative of Switzerland for sharing the views of his delegation and making proposals with regard to the text. Next on my list is Brazil, to be followed by Article 36. But let me first pass the floor to the distinguished representative of Brazil. You have the floor.

Thank you, Mr. Chair, and good morning, everyone. With regard to box four, my delegation once again believes that the text before us is very close to a consensus worthy formulation and that it only merits some fine tuning. My delegation in particular supports the proposal made by the delegation of Norway on paragraph two to give more granularity to the reference to the life cycle of laws by adding references to its various phases, such as design, development and deployment. On paragraph 3, my delegation concurs with the proposal by Russia, supported by others, to eliminate the word "properly". On paragraph 4, like other delegations, we believe that the language merits reformulation. We believe that the purpose of this paragraph is to clarify that the additional reviews are to be triggered by changes to the critical functions of existing laws.

We therefore propose the following alternative language:.

And I'll read it slowly, ensure that any further development or modification to existing lethal autonomous weapons systems that have an impact on its target selection and engagement functions or on its intended effects should be subject to further review in order to ensure compliance with IHL.

Naturally, we will send this language to you, Mr. Chair, and to the Secretariat.

On paragraph 6, my delegation concurs that the issues of data bias and automation bias are of different natures.

And that there is a need.

To transpose the reference to automation bias to a separate paragraph, as proposed by the ICRC. In our view, this new paragraph should also point towards possible measures to that effect, such as in the areas of human-machine interaction, doctrine, and training. My delegation also supports the proposal by the delegation of Belgium regarding the insertion of a new paragraph.

On the need to prevent diversion of laws to unauthorized end-users.

I thank you, Chair.

Let me thank the distinguished representative of Brazil for sharing his views and comments and making proposals and for sending the text to the Chair. Next on my list is Article 36, and then I have Bulgaria and Greece. But I will first pass the floor to the representative of Article 36. You have the floor.

Thank you, Chair. I just want to make some broad comments on this section of the text. Well, the first one isn't broad; it's a little bit specific. But I feel some uneasiness with the term "lifecycle" that's used here. I know we've committed ourselves to avoiding anthropomorphizing. I feel like "lifecycle" is borderline anthropomorphizing, maybe zoomorphizing. I'd prefer it if we -- it would be perhaps better if we could use the more precise language that appears in paragraph 1 already. In general, I feel that paragraph 3, with its commitment to adequately and properly understand the systems that are going to be used, is really important in the context of autonomous weapons systems. Structurally, I feel that that paragraph could be brought up in the text as a more primary commitment, and some of the other obligations in that section could be brought under it and in relationship to it. And perhaps at the same time, feel like the weapon review commitment could be brought down in the text rather as a sort of provision that wraps up the orientation to the obligations that would come before it through the text. So it's just a bit of a structural feeling, perhaps, for the flow of this. Certainly appreciate the inputs from states who've been calling for this section to be expressed more strongly as well in relation to the chapeau. seems to be quite a substantial engagement recognition that these different forms of bias present themselves differently, with automation bias being tied to the sufficient application of human judgment, whereas bias in data sets and algorithms and AI systems is rather a different phenomenon. On that latter form of bias, bias in data sets and algorithms, a little bit wary that perhaps we can be over-sanitizing our conversation on this. I think for us, these forms of bias are particularly a problem, particularly challenging when we're talking about systems that would target people, systems that use target profiles that represent human beings, particularly in systems that might purport to distinguish targetable groups of people from non-targetable groups of people. We see this as deeply problematic systems that would function in such a way. I mean, we live in a society where power structures of racism and associations of masculinity and violence are still strongly embedded. They're embedded in our societies, they're embedded in the data sets that we produce and the technologies that we produce. And it's really the potential for those phenomenon to be replicated and to present themselves through these weapon systems that is of distinct concern for us. If we were to review our systems and find that they target people of color more than they target white people, for example, this would seem deeply troubling. And it's not clear to me that the types of provisions we have on the screen here are really an adequate response to that form of replication of racism in society, for example. This is one of the reasons why we would support, in general, prohibition on systems that target people. The concern of bias in those systems is very important to us, and I think this isn't an issue that we've really wrestled with in our discussions here adequately enough. I put these thoughts on the table to perhaps allow us to come back to that a bit more in the future. Thanks, Joe.

Let me thank the distinguished representative of Article 36 for sharing his views. And next on my list is Bulgaria. And then I have three more, Greece, Sweden, China. But let's see. It's eight minutes to one. So first, the distinguished representative of Bulgaria.

Thank you, Chair. Bulgaria supports box four as a whole, as it contains important measures to ensure compliance with IHL. One such measure is the conduct of legal reviews of autonomous weapon systems, highlighted in para one and four.

However, legal reviews also.

Happen to be a legal obligation to those states parties to Additional Protocol I to the Geneva Conventions. So as others mentioned earlier, should is probably not the right verb in the support of box 4 in relation to paras 1 and 4. And proposals made by April, Italy, Norway, and Austria may serve as fixes to the situation. We also appreciate the inclusion.

Of the second sentence in para one as as we have always.

Been a strong proponent for on sharing best practices on para six we support the proposal made by the ICRC to separate data sets bias from automation bias in different paragraphs or sub paragraphs as you to make the language clearer and more digestible for delegations thank you.

I would like to thank the distinguished representative of Bulgaria for sharing his views and making proposals. Next on my list is Greece. Sir, you have the floor.

Thank you, Mr. Chair. We would like to provide some elements of consideration regarding box four. We support paragraphs one and two as they stand. Regarding paragraph three, we believe the issue outlined are already covered under paragraph five And as such, we also propose the deletion of paragraph three. In paragraph four, we believe the aspect of development is addressed in paragraph one. Therefore, we suggest that this paragraph should be focused solely on the modification of critical operational capabilities of laws. Finally, in paragraph eight, we recommend incorporating the elements of guidance and instructions into part five by adding the following wording at its end. and subsequently provide the human user with guidance and instructions based on their results.

Thank you.

Let me thank the distinguished representative of Greece for making some proposals. Next on my list is Sweden to be follow- no, Sweden. You have the floor.

Thank you, Chair.

Sweden too wishes to express support for this box as a whole. We believe that it includes measures that are crucial to enable and ensure respect for and compliance with international law, in particular IHL.

We think it could be beneficial to make some small tweaks, but we consider also, but we could also support the box as it currently is formulated.

So my comments are mostly reactions to proposals put forward by other delegations. Regarding reformulations, we can support the term "unintended bias" and a merge of para 6 and 7 as proposed by Australia. We also appreciate and can support the proposal made by the Asia-Pacific Institute for Law and Security on reformulation of para 4. We can also support the proposal to change "properly" to "sufficiently understood" in para 3 as proposed by France. And regarding the chapeau and the shall, must, should discussion, we can support a shall in the chapeau for or within Paris 1 to 4, provided that voluntarily remains in regard to voluntary exchange of best practice. But we prefer Paris 5 to 7 to maintain a should or be under a chapeau that includes should. We also support Switzerland's proposal that para 8 reflects existing IHL and can thus be formulated with a shall.

Thank you, Chair.

Let me thank the distinguished representative of Sweden, especially with regard to the use of operational verbs. Next on my list is China, and then I have Mexico and stop killer robots, but that will most likely be for after lunch. But let me first pass the floor to the distinguished representative of China. Madam, you have the floor.

Thank you, Mr. Chair. This box mainly concerns the control measures, entire life cycle, legal review, testing, elimination of prejudice, bias, training, and important critical points. This lays a good basis for discussion. We believe that in order to reach agreement, It's necessary to properly handle the balance between specific requirements and the control measures taken by states. It's also necessary to take into account the differences in technical capabilities of various states. On the basis of 11 guiding principles of 2019 and the report of GGE in 2023, We should find scientific and effective criteria and requirements to guide countries to establish effective control system. At the same time, we should avoid the situation in which inflexible rules and regulations can impede the technical innovation and proper use.

Of technology by countries.

Concerning specific formulation of paragraphs, it's necessary to highlight the basic requirements of key issues and leave the space to states to decide on the specific.

Measures to be taken.

Concerning wording, we have the following suggestion. We agree with the proposal by some countries that para 1 and para 4 duplicate each other to some extent. We suggest a reformulation of them. We also support the suggestion to merge para 6 with para 7. Besides, in para 1, it mentions the encouragement of exchange of experience. We suggest that at the end of this box, we add one paragraph, which focuses on the cooperation in this area, including the encouragement to various countries to improve exchange on oversight and regulation of laws.

At the same time.

It's necessary to remove scientific technological barriers so as to ensure that all countries can fully enjoy the right to technical development.

We will submit our wording to the secretariat.

Thank you, sir.

I would like to thank the distinguished representative of China for her comments and proposals, and indeed we would like to receive the text for such a para also in, in writing for proper consideration. It's, uh, one o'clock, uh, many thanks for the interpreters for, uh, going beyond one o'clock, um, we have, uh, one, uh, announcement, um. I have two events to be announced during the lunch break. The first one is a briefing organized for delegates whose participation in this session of the GDE on laws is supported by the CCW sponsorship program, and it will be held at 1:15 in Room 23, which is to the right of this conference room. And the second one is a side event on bias in automatic um in autonomous weapon systems and compliance with IHL addressing a missing link this side event is organized by the Stockholm International peace research Institute cpr and will also be held at 1:15 in but in room 21 which is located in this building on the same floor I wish you all a good lunch see you back at 3:00 um still to hear the remaining interventions from delegations and then to move over to box five. I think we are on schedule. Thank you very much.