Most organizations today are exploring the use of emerging AI-powered technologies to improve their workflows and processes, analyze and summarize data, and generate content faster than ever before. You and your coworkers may already be using AI tools and frameworks for activities such as conducting research, generating content, and solving coding challenges.
However, it’s important to temper your organization’s enthusiasm for AI with appropriate guidance and restrictions. A wholesale ban on AI technologies could undermine business needs and goals, so how can you be vigilant about protecting sensitive data and ensuring that generated content is accurate? The solution starts with developing an AI security policy.
A strong AI security policy will empower your organizations to reap the benefits of AI, while prescribing the risk assessments and security controls necessary to protect sensitive data and ensure content accuracy. At the same time, it’s essential to ensure that your vendors, suppliers and other third parties have security controls that can sufficiently protect your critical systems and data.
In this post, I examine AI security policies; review how AI security policies can be applied to third parties; and share key questions to assess your vendors’ and suppliers’ AI security controls.
An AI security policy is a set of guidelines for evaluating and using artificial intelligence tools and frameworks in a way that maximizes insight, control, and data protection. It also outlines AI vulnerabilities and presents measures to mitigate their potential risks.
With a well-designed AI security policy, an organization can safeguard the security and integrity of its AI systems and any data handled by artificial intelligence. AI policies typically include provisions for:
An AI security policy is typically an extension of an organization’s general information security policy and associated controls, with some shared concepts regarding data protection, privacy and accuracy.
These policies specify workflows and procedures for security teams who need to evaluate AI tools for use at their organizations. They outline required levels of data protection for different types of services and indicate procedures for reviewing how sensitive data may be used as AI training inputs and/or appear in AI-generated content.
Source code policies emphasize secure development practices (e.g., adhering to coding standards, conducting regular code reviews) and specify monitoring and logging mechanisms for tracking system behavior and detecting anomalies. These policies often require the organization to track the use of source code as an input to AI tools, as well as the use of any code generated by AI.
Incident response policies outline protocols for handling security breaches and emphasize compliance with industry standards and regulations calling for such protocols.
Data retention policies ensure that input data uploaded to AI tools and services is deleted within an acceptable timeframe. Privacy considerations must also be evaluated, as some regions have temporarily banned the use of generative AI tools due to concerns around the collection and use of personal data.
Ethical considerations and awareness training are included in AI security policies to address biases, ensure accountability, and foster a culture of security. For example, they may require users of generative AI to review and edit created content for accuracy, bias, and offensive content.
AI security policies should acknowledge that AI tools have been known to produce incorrect, biased, or offensive results. Of particular concern are “AI hallucinations,” which occur when generative AI tools create content that is unexpected, untrue, or not backed up by evidence and real-world data. AI tools also tend to have a limited knowledge of recent real-world events, which can lead to further inaccuracies and omissions in generated content.
Your AI security policy should include standards for evaluating, mitigating, and monitoring the risks for all AI solutions that process or generate data for your organization – including those provided and/or used by your third-party vendors, suppliers, and service providers. It’s critical to ensure that third-party tools and services protect sensitive data, sanitize inputs to remove confidential information, and follow other required security controls.
There are three primary ways to leverage AI security policies in your third-party risk management (TPRM) program: pre-contract due diligence, vendor contracting, and vendor assessment.
Your AI security policy should guide the due diligence process when evaluating potential vendors and suppliers. By referencing the policy, your organization can systematically assess a vendor's security controls, data protection mechanisms, and access protocols. This minimizes potential vulnerabilities by ensuring that external parties meet the same rigorous security criteria as that applied to your internal systems.
Contractual agreements with vendors and suppliers can be informed by AI security policy provisions. By incorporating policy guidelines into agreements, your organization sets clear expectations regarding security requirements, data handling practices, and incident response procedures. This alignment ensures that vendor-provided AI solutions or services uphold the organization's security standards, contributing to a more secure and resilient AI ecosystem.
When used as a part of vendor assessments, your AI security policy can act as a reference point for gauging their security practices against your organization's defined standards. This also ensures consistency in setting security expectations across your vendor ecosystem.
Overall, an AI security policy acts as a comprehensive framework for evaluating and aligning vendor and supplier security practices with your organization's strategic AI objectives.
Hidden threats can lurk within third-party AI providers, posing risks that might not be immediately evident. These threats encompass security vulnerabilities, potential data breaches, covert malicious code, data misuse, and biases in algorithms – each of which could compromise your organization's data, reputation, and operations.
To counter these risks, your organization must conduct diligent evaluations of its third-party providers. This due diligence should assess security measures, data protection practices, and algorithmic transparency. Continuous monitoring, robust contractual agreements, and contingency planning are also critical to revealing and mitigating hidden AI threats.
In December 2021, Microsoft released an AI security risk assessment framework to help organizations audit, track and improve the security of the AI systems. Prevalent built on this framework to create a 16-question survey that you can use to assess the AI security controls employed by your vendors and suppliers.
Use this third-party AI security assessment to:
Questions | Possible Responses |
---|---|
1. Is data collected for the AI system(s) from trusted sources? |
Select one: a) Yes, the organization ensures that data collected is received from trusted sources only. b) Data collected is not exclusively from trusted sources. c) Not applicable. |
2. Has a data policy been developed that includes the privacy and protection of sensitive data types? |
Select all that apply: a) A formal data policy has been established. b) The data policy is communicated and made available to all personnel involved with the use or creation of AI systems. c) Data is classified using data classification labels. d) None of the above. |
3. Does the organization ensure the secure storage of data based on its classification? |
Select all that apply: a) The organization utilizes secure storage, based on a defined data classification process. b) Data used in AI systems is classified and protected based on a defined classification policy. c) Access to data is audited, and formal user access request approval is enforced. d) Datasets are version controlled and follow defined change control processes. e) None of the above. |
4. Are datasets appropriately tracked and verified via cryptographic hash before use? |
Select all that apply: a) The organization enforces role-based access control for datasets. b) Access audits are performed on a regular basis. c) Steps are implemented to ensure any third-party resource provider or external parties cannot access test data assets. d) None of the above. |
5. How does the organization ensure data integrity remains throughout an AI system lifecycle? |
Select all that apply: a) Unique identification is applied to datasets. b) A central location is used to track datasets and their cryptographic descriptions. c) Access to datasets is audited periodically. d) changes made to datasets go through management approval before submission. e) None of the above. |
6. Are data processing pipelines secured? |
Select one: a) Yes, the organization takes steps to ensure the security of its data processing arrangements. b) No, the organization does not take steps to ensure the security of its data processing arrangements. c) Not applicable. |
7. Does the organization secure subsets of data in the same manner as datasets? |
Select one: a) Yes, the organization applies the same level of security and data categorization processes for data subsets. b) No, the organization does not apply the same level of security and data categorization processes for data subsets. c) Not applicable. |
**8. Does the organization review its model training code within an appropriate environment? |
Select one: a) Yes, the organization reviews and manages model code in secure and dedicated environments away from production. b) Model code is not formally reviewed within dedicated environments away from production. c) Not applicable. |
9. Is training for the model used in the AI system conducted under the same conditions that would be found at deployment? |
Select one: a) Yes, the organization ensures that training the model is conducted in the same conditions as when at deployment stage. b) The organization does not conduct training of the model in the same conditions as when at deployment stage. c) Not applicable. |
10. Does the model design and training algorithm include explicit or implicit model regularization? |
Select one: a) Yes, regularization is ensured for the model design and training algorithms. b) No, regularization is not ensured for the model design and training algorithms. c) Not applicable. |
11. Are models continuously retrained as new training data flows into training pipelines? |
Select one: a) Yes, the organization has established a continual retraining program to test and validate new training data. b) No, the organization has not established a continual retraining program to test and validate new training data. c) Not applicable. |
12. How does the organization secure its AI systems prior to deployment? |
Select one: a) Formal acceptance testing criteria have been defined and documented for new AI systems, upgrades, and new versions. b) New AI systems, upgrades or new versions are implemented with formal testing. c) The organization leverages automated tools for testing information systems, upgrades, or new versions. d) Test environments closely resemble the final production environment. e) The frequency, scope, and method(s) for independent security reviews is documented. f) None of the above. |
13. How does the organization secure and manage the underlying network where the AI system resides? |
Select one: a) Gateway devices are to filter traffic between domains and block unauthorized access. b) Secure configuration guidelines are documented and periodically reviewed. c) Networks are segregated, in line with a defined access control policy. d) Requirements are established to segregate and restrict use of publicly accessible systems, internal networks, and critical assets. e) None of the above. |
14. How does the organization log and monitor its AI systems and supporting infrastructure? |
Select one: a) The organization has an event logging system in place (e.g., a SIEM solution) for monitoring event and security logs. b) Event and security logs are reviewed regularly for abnormal behavior. c) Consolidated reports and alerts are produced and reviewed for system activity. d) Logging and monitoring include storage, pipelines, and production servers. e) None of the above. |
15. How does the organization identify and manage security incidents related to, or impacting AI systems? |
Select one: a) A formal process is established for reporting AI systems incidents. c) Formal incident response and escalation procedures have been established. d) Incident response procedures are tested on a periodic basis. e) None of the above |
16. Does the organization have processes to ensure that AI systems can be remediated and recovered after an incident? |
Select one: a) Critical AI assets are identified and inventoried. b) The organization has developed a formal business continuity plan. c) Impact assessments include planning for the impact of losing critical AI systems to attacks. d) Business continuity testing is conducted and occurs on a repeated schedule for critical AI systems. e) None of the above. |
Use the above questionnaire as a starting point to uncover risks in the AI systems employed by your vendors and suppliers. By proactively identifying and managing third-party AI risks, you can protect your organization’s systems and data while avoiding potential issues related to fairness, transparency and accountability. Third-party AI risk assessments not only safeguard your operations, but also aid in ethical decision-making, vendor selection, and long-term partner relations.
For more information about how Prevalent can assist your organization in assessing vendor and supplier AI security in the context of overall third-party risk, request a demo today.
Prevalent continues to set the pace in third-party risk management with customer-focused enhancements that simplify the...
06/12/2024
The European Union today approved sweeping AI regulations, set to go into effect in 2026. Here...
03/13/2024
World governments and standards bodies have started to respond to AI technologies with new compliance regulations...
01/04/2024