What Procurement Teams Need to Know When Buying AI Tools

Why AI Procurement Is Different From Traditional Software Procurement

Traditional software procurement evaluates functionality, price, integration, and support. AI tool procurement requires all of these plus a set of requirements unique to AI: how the vendor handles input data, whether inputs are used for model training, what data retention policies apply, who has access to the data, where it is processed and stored, and what happens to the data when the contract ends. These questions do not arise in traditional software procurement because traditional software does not learn from and potentially redistribute user inputs.

The procurement team is often the first organizational function to engage with an AI vendor. If procurement does not ask the right questions during evaluation, the organization may sign a contract that creates governance problems: data is retained longer than acceptable, used for training in ways that violate the AI policy, or processed in jurisdictions that create regulatory exposure. Fixing these issues after contract execution is expensive and often impossible.

This guide covers the eight procurement responsibilities for AI governance, the questions auditors will ask about your procurement process, the five most common procurement mistakes, evaluation criteria for AI vendors, and how PolicyGuard supports procurement. For the broader governance framework, see our complete AI policy and governance guide.

Your Core AI Governance Responsibilities as Procurement

• AI tool vendor questionnaire process: Procurement must use an AI-specific vendor questionnaire that addresses data handling, model training, data retention, subprocessor management, and regulatory compliance. The standard vendor questionnaire is insufficient for AI tools. Failure looks like signing a contract without knowing whether the vendor uses customer data for model training. See our AI governance toolkit for questionnaire templates.
• Security and compliance certification verification: Procurement verifies that AI vendors hold relevant certifications: SOC 2 Type II, ISO 27001, GDPR compliance attestations, and any industry-specific certifications. Self-reported compliance is insufficient; procurement should request certification evidence. Failure means relying on vendor marketing claims without verification.
• DPA and contract negotiation: Every AI tool that processes organizational data requires a Data Processing Agreement. Procurement negotiates DPA terms including data retention limits, training data opt-out, subprocessor management, breach notification timelines, and data deletion procedures. Failure means data being processed under the vendor's standard terms, which typically favor the vendor. See our GDPR generative AI compliance guide.
• Approved tool list coordination with IT: Procurement coordinates with IT to ensure purchased AI tools are added to the approved tool list and that governance controls are configured before employees are given access. Failure means employees accessing newly purchased AI tools before governance controls are in place.
• AI vendor risk tiering and ongoing monitoring: Not all AI vendors present the same risk. Procurement tiers vendors by risk level based on data sensitivity, usage volume, and criticality, then applies ongoing monitoring proportionate to the tier. Failure means treating all AI vendors equally, resulting in over-assessment of low-risk vendors and under-assessment of high-risk ones.
• Budget management for AI governance tools: Procurement manages the budget for AI governance tools, ensuring spending aligns with organizational priorities and provides value proportionate to cost. Failure means governance tool spending without clear budget ownership or ROI tracking.
• Renewal review and vendor performance: At contract renewal, procurement reviews vendor performance against governance requirements: Have they maintained certifications? Have they changed data handling practices? Have they added subprocessors? Failure means automatically renewing contracts without verifying the vendor still meets governance requirements. See our shadow AI risk guide for understanding vendor risks.
• New AI tool intake process coordination: Procurement coordinates the intake process when employees request new AI tools, ensuring requests are routed through the appropriate evaluation workflow involving IT, security, compliance, and legal. Failure means employees purchasing AI tools through departmental budgets without governance review. See our what is AI governance guide for foundational concepts.

The Questions Your Board, Auditors, or Regulators Will Ask You

"What process does procurement follow before approving a new AI tool?"

Evidence includes the procurement evaluation workflow, the AI-specific vendor questionnaire, evaluation criteria, and records of completed evaluations. Without a documented process, this question reveals ad hoc procurement that creates governance risk.

"What contracts govern data handling with AI vendors?"

Evidence includes executed DPAs, contract terms for data handling, and a register of vendor data processing practices. Without DPAs, organizational data may be processed under vendor standard terms that do not meet governance requirements.

"How do you assess AI vendor security and compliance certifications?"

Evidence includes the certification verification process, records of certifications reviewed, and any gaps identified. Auditors want to see that procurement verified certifications rather than accepting vendor claims.

"What happens when an AI vendor has a security incident?"

Evidence includes the vendor incident response requirements in the contract, notification timelines, and the organization's response procedures for vendor incidents.

"How do you ensure AI tools purchased comply with the organizational AI policy?"

Evidence includes the evaluation criteria that map to AI policy requirements, records showing each purchased tool was evaluated against these criteria, and the approved tool list process. See our SaaS AI governance guide for vendor-specific considerations.

The 5 Biggest Mistakes Procurement Teams Make on AI Governance

1. Evaluating AI tools on functionality alone without compliance involvement

Procurement teams evaluate AI tools primarily on what the tool does, how much it costs, and how well it integrates with existing systems. Compliance, legal, and security involvement comes later, if at all. This is a critical sequencing error because by the time governance stakeholders engage, the procurement team has already invested time in the evaluation, formed vendor relationships, and may have even signed a letter of intent. Governance concerns raised at this stage feel like obstacles rather than requirements, and there is organizational pressure to proceed despite identified gaps. The cost is purchasing tools that create compliance problems that are expensive to fix after deployment. The fix is involving compliance, legal, and security at the evaluation stage, not the contracting stage. A cross-functional evaluation checklist ensures governance requirements are assessed alongside functional requirements from the beginning.

2. Signing contracts before DPA negotiation

Many AI vendors include data processing terms in their standard terms of service. If procurement signs the contract before negotiating a DPA, the organization is bound by the vendor's standard terms, which typically include broad data usage rights, long retention periods, and limited subprocessor transparency. Negotiating a DPA after contract execution is difficult because the vendor has no incentive to make concessions. The cost is ongoing data processing under terms that do not meet the organization's governance requirements, with limited ability to change them until contract renewal. The fix is requiring DPA negotiation as a condition precedent to contract execution. The DPA should be finalized before the main contract is signed, not after.

3. No ongoing vendor monitoring after initial procurement

Procurement evaluates the AI vendor at the time of purchase and then does not reassess until contract renewal. During the contract period, the vendor may change data handling practices, add subprocessors in new jurisdictions, let certifications lapse, or modify terms of service. Without ongoing monitoring, these changes go unnoticed until they create problems. The cost is vendor risk that increases without the organization's awareness, potentially creating compliance violations that only become apparent during an audit or incident. The fix is a risk-tiered vendor monitoring program that reassesses high-risk vendors at least annually and monitors all vendors for material changes to certifications, subprocessor lists, and data handling practices.

4. Not checking AI vendor subprocessor lists

AI vendors often use subprocessors for infrastructure, model hosting, and data processing. These subprocessors have their own data handling practices and may be located in jurisdictions with different regulatory requirements. Procurement teams that do not review subprocessor lists miss a critical risk factor: the organization's data may be processed by entities that have never been assessed and may not meet governance requirements. The cost is unknown data exposure through the vendor's supply chain. The fix is requiring subprocessor lists as part of the evaluation, reviewing each subprocessor's location and data handling capabilities, and negotiating notification requirements for subprocessor changes in the DPA.

5. No process for employees requesting AI tools outside of formal procurement

When employees need an AI tool quickly and there is no fast intake process, they purchase it themselves using departmental budgets, personal credit cards, or free tier accounts. These purchases bypass procurement entirely, meaning no vendor assessment, no DPA, no certification verification, and no governance control configuration. By the time procurement becomes aware of the tool, it is already in use with organizational data. The cost is a growing inventory of ungoverned AI tools purchased outside the procurement process, each representing unassessed vendor risk. The fix is creating a fast-track AI tool intake process that employees can use to request new tools, with a target turnaround of five to ten business days for basic evaluation. Speed reduces the incentive for employees to bypass the process.

What to Look For When Evaluating AI Governance Tools

• SOC 2 Type II certification: This is the minimum security certification for any AI tool processing organizational data. Verify the certification is current and covers the specific services being purchased. Red flags include vendors with only SOC 2 Type I or no certification at all.
• GDPR and data protection compliance posture: For organizations subject to GDPR, the vendor must demonstrate compliance including data subject rights support, lawful basis documentation, and cross-border transfer safeguards. Red flags include vendors that cannot articulate their GDPR compliance approach.
• Data retention and deletion policies: Understand exactly how long the vendor retains input data and whether it can be deleted on request. Good looks like configurable retention with verified deletion. Red flags include indefinite retention or no deletion capability.
• Subprocessor transparency: Good looks like a published, current subprocessor list with notification commitments for changes. Red flags include no subprocessor visibility or refusal to disclose.
• Incident notification commitments: Good looks like contractual commitment to notify within 24 to 72 hours of a security incident affecting your data. Red flags include no notification timeline or "reasonable time" language.
• Contract flexibility and exit provisions: Good looks like clear data export and deletion procedures at contract end, with reasonable termination provisions. Red flags include lock-in terms, data hostage scenarios, or unclear data handling at termination.

How PolicyGuard Helps Procurement Teams Specifically

• AI tool discovery for procurement visibility: PolicyGuard gives procurement visibility into what AI tools are actually being used across the organization so you can identify tools that were purchased outside formal procurement and bring them into the governance process.
• Vendor risk assessment support: PolicyGuard provides AI-specific vendor risk assessment templates and tracking so procurement can evaluate and monitor AI vendors with criteria that address the unique risks AI tools present.
• Approved tool list management: PolicyGuard maintains the approved AI tool list that procurement helps populate, ensuring purchased tools are immediately reflected in the governance framework and available to employees through the approved channel.
• Shadow procurement detection: PolicyGuard detects when employees use AI tools that have not been procured through the formal process, giving procurement visibility into unauthorized purchases that need to be brought into compliance.
• Audit evidence for procurement governance: PolicyGuard generates audit evidence documenting the AI tool procurement process, vendor assessments, and ongoing monitoring, demonstrating to auditors that procurement governance is operating effectively. Start your free trial to see the procurement visibility features.

Frequently Asked Questions

What questions should procurement ask AI tool vendors before buying?

Procurement should ask: How is our input data stored, processed, and retained? Is our data used to train or improve your models? Who are your subprocessors and where are they located? What security certifications do you hold and are they current? What is your breach notification timeline? Can we configure data retention and deletion? What happens to our data when the contract ends? Can you comply with our Data Processing Agreement requirements? These questions address the unique risks AI tools present beyond traditional software.

What contracts and agreements are needed when purchasing AI tools?

At minimum, procurement needs a Data Processing Agreement covering data handling, retention, and deletion; a master service agreement with liability and indemnification provisions; a subprocessor addendum with notification requirements; a security addendum or SOC 2 report; and a data deletion certificate procedure for contract termination. For tools processing EU personal data, Standard Contractual Clauses may also be required.

How do you assess AI vendor security and compliance certifications?

Request certification evidence directly from the vendor: SOC 2 Type II reports, ISO 27001 certificates, and any sector-specific certifications. Verify certifications are current and cover the specific services being purchased. Review the SOC 2 report for exceptions or qualifications. Check whether the certification scope includes the AI processing services, not just the vendor's infrastructure. Follow up annually to confirm certifications are maintained.

What red flags should procurement look for in AI vendor pitches?

Red flags include vendors that cannot clearly explain their data handling practices, vendors that resist DPA negotiation, vendors without SOC 2 Type II or equivalent certification, vendors that will not disclose their subprocessor list, vendors with vague or evasive answers about data retention, vendors that claim compliance without evidence, and vendors whose terms of service include broad data usage rights with no opt-out. Any of these red flags warrants additional scrutiny or vendor elimination.

How does procurement coordinate with IT, Legal, and Compliance on AI tool purchases?

Procurement coordinates through a cross-functional evaluation process: IT assesses technical integration and security, Legal reviews contracts and DPAs, Compliance evaluates regulatory alignment, and Procurement manages the vendor relationship and negotiation. A standardized evaluation checklist ensures all perspectives are captured before a procurement decision is made. The key is involving all stakeholders during evaluation, not after contract execution.

This week, take three actions: review your current vendor questionnaire to determine whether it includes AI-specific questions about data handling, model training, and subprocessors, check whether all current AI vendor contracts include executed DPAs with adequate terms, and identify any AI tools being used across the organization that were not purchased through formal procurement. If any of these areas has gaps, PolicyGuard can help you identify and manage ungoverned AI tools.