Talking terms: Agent information in LLM supply chain bargaining
Corresponding Author
Samuel N. Kirshner
UNSW Business School, UNSW Sydney, Sydney, Australia
Correspondence
Yiwen Pan, College of Economics, Zhejiang Gongshang University, Hangzhou, China.
Email: [email protected]
Search for more papers by this authorYiwen Pan
College of Economics, Zhejiang Gongshang University, Hangzhou, China
Search for more papers by this authorJason Xianghua Wu
UNSW Business School, UNSW Sydney, Sydney, Australia
Search for more papers by this authorCorresponding Author
Samuel N. Kirshner
UNSW Business School, UNSW Sydney, Sydney, Australia
Correspondence
Yiwen Pan, College of Economics, Zhejiang Gongshang University, Hangzhou, China.
Email: [email protected]
Search for more papers by this authorYiwen Pan
College of Economics, Zhejiang Gongshang University, Hangzhou, China
Search for more papers by this authorJason Xianghua Wu
UNSW Business School, UNSW Sydney, Sydney, Australia
Search for more papers by this authorAbstract
We investigate the use of large language models as agents (LLM agents) in autonomous supply chain contract negotiations. Our objectives are to assess whether LLM agents exhibit human-like bargaining behaviors and to explore the impact of information on performance. To address these objectives, we conducted several experimental studies using LLM agents as participants and compared the results with human results from a benchmark study. Our experiments covered scenarios where supplier cost information was public, private, ambiguous, or deceptive. Overall, we found that LLM agents use simple heuristics to make decisions and generally exhibit human-like negotiating behavior. Contrasting humans, LLM agents are more inclined toward reaching agreement, leading to greater supply chain efficiency but potentially greater inequality compared to human negotiators. Deceiving LLM agents into believing they have higher costs can improve outcomes for the supplier at the expense of retailers and the supply chain's efficiency. We also show that tailored retrieval-augmented generation (RAG) configurations can enhance negotiation outcomes. Taken together, our results (1) provide timely insights into the integration of AI into supply chains, (2) raise ethical questions around the trade-off between inequality and efficiency and the use of deception with LLM agents, (3) highlight the effectiveness of tailoring RAG configurations to optimize specific objectives such as efficiency or stakeholder profitability, and (4) provide many avenues for future research into examining LLM agents as supply chain negotiators.
Supporting Information
Filename | Description |
---|---|
deci70010-sup-0001-SuppMat.pdf1,005.2 KB | Figure S1 Total bargaining periods for LLM agents with different ending designs. Table S1 Supply chain efficiency (%) comparison across experimental designs. Table S2 Wholesale price (w) comparison across experimental designs. Table S3 Order quantity (q) comparison across experimental designs. Figure S2 The concession process: Supplier's share of expected supply chain profits (%) by endings. Figure S3 Supplier share of supply chain expected profits (%) under different models. Table S4 Observed expected supply chain efficiencies (%) across models. Table S5 Ordered quantity and wholesale price across models under public information. Table S6 Ordered quantity and wholesale price across models under private information. Table S7 Observed expected supply chain efficiencies (%) Figure S4 The concession process: Supplier's share of expected supply chain profits (%). |
Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.
REFERENCES
- Abdelnabi, S., Gomaa, A., Sivaprasad, S., Schönherr, L. & Fritz, M. (2023) LLM-deliberation: evaluating LLMs with interactive multi-agent negotiation games. arXiv preprint arXiv:2309.17234.
- Abolghasemi, M., Ganbold, O. & Rotaru, K. (2023) Humans vs large language models: judgmental forecasting in an era of advanced AI. arXiv preprint arXiv:2312.06941.
- Becker-Peth, M., Katok, E. & Thonemann, U.W. (2013) Designing buyback contracts for irrational but predictable newsvendors. Management Science, 59(8), 1800–1816.
- Bianchi, F., Chia, P.J., Yuksekgonul, M., Tagliabue, J., Jurafsky, D. & Zou, J. (2024) How well can LLMs negotiate? NegotiationArena platform and analysis. arXiv preprint arXiv:2402.05863.
- Binz, M. & Schulz, E. (2023) Using cognitive psychology to understand GPT-3. Proceedings of the National Academy of Sciences, 120(6), e2218523120.
- Browne, R. (2023) An AI just negotiated a contract for the first time ever - and no human was involved. CNBC. Available at: https://www.cnbc.com/2023/11/07/ai-negotiates-legal-contract-without-humans-involved-for-first-time.html.
- Chen, F., Zhao, Y. & Thonemann, U. (2023) The value of response time information in supply chain bargaining. Manufacturing & Service Operations Management, 25(1), 19–35.
- Chen, Y., Kirshner, S.N., Ovchinnikov, A., Andiappan, M. & Jenkin, T. (2025) A manager and an AI walk into a bar: does ChatGPT make biased decisions like we do? Manufacturing & Service Operations Management, 27(2), 354–368.
10.1287/msom.2023.0279 Google Scholar
- Chen, Y., Liu, T.X., Shan, Y. & Zhong, S. (2023) The emergence of economic rationality of GPT. Proceedings of the National Academy of Sciences, 120(51), e2316205120.
- Cromwell, J., Harvey, J., Haase, J. & Gardner, H. (2023) Discovering where ChatGPT can create value for your company. Harvard Business Review. Available at: https://hbr.org/2023/06/discovering-where-chatgpt-can-create-value-for-your-company.
- Davis, A.M. (2015) An experimental investigation of pull contracts in supply chains. Production and Operations Management, 24(2), 325–340.
- Davis, A.M. & Hyndman, K. (2019) Multidimensional bargaining and inventory risk in supply chains: an experimental study. Management Science, 65(3), 1286–1304.
- Davis, A.M. & Hyndman, K. (2021) Private information and dynamic bargaining in supply chains: an experimental study. Manufacturing & Service Operations Management, 23(6), 1449–1467.
- Davis, A.M. & Leider, S. (2018) Contracts and capacity investment in supply chains. Manufacturing & Service Operations Management, 20(3), 403–421.
- Donohue, K., Özer, Ö. & Zheng, Y. (2020) Behavioral operations: past, present, and future. Manufacturing & Service Operations Management, 22(1), 191–202.
- Falcão Filho, H.A. (2024) Making sense of negotiation and AI: the blossoming of a new collaboration. International Journal of Commerce and Contracting, 8(1–2), 44–64.
10.1177/20555636241269270 Google Scholar
- Fu, Y., Peng, H., Khot, T. & Lapata, M. (2023) Improving language model negotiation with self-play and in-context learning from AI feedback. arXiv preprint arXiv:2305.10142.
- Galinsky, A.D. & Mussweiler, T. (2001) First offers as anchors: the role of perspective-taking and negotiator focus. Journal of Personality and Social Psychology, 81(4), 657.
- Goli, A. & Singh, A. (2024) Can large language models capture human preferences? Marketing Science, 43, 709.
10.1287/mksc.2023.0306 Google Scholar
- Huang, K. & Zhu, F. (2023) ChatGPT and gig economy. In: K. Huang, Y. Wang, F. Zhu, X. Chen & C. Xing (Eds.) Beyond AI: ChatGPT, Web3, and the business landscape of tomorrow, Cham: Springer, pp. 129–158.
10.1007/978-3-031-45282-6_5 Google Scholar
- Hyndman, K.B. & Menezes, M.B. (2023) Behavioral pitfalls of product proliferation in supply chains: an experimental study. Decision Sciences, 54(2), 131–153.
- Jackson, I., Ivanov, D., Dolgui, A. & Namdar, J. (2024) Generative artificial intelligence in supply chain and operations management: a capability-based framework for analysis and implementation. International Journal of Production Research, 62, 6120.
- Katok, E. & Wu, D.Y. (2009) Contracting in supply chains: a laboratory investigation. Management Science, 55(12), 1953–1968.
- Kirshner, S. (2024) Artificial agents in operations management experiments. SSRN 4726933.
- Kwon, D., Weiss, E., Kulshrestha, T., Chawla, K., Lucas, G.M. & Gratch, J. (2024) Are LLMs effective negotiators? Systematic evaluation of the multifaceted capabilities of LLMs in negotiation dialogues. arXiv preprint arXiv:2402.13550.
- Leider, S. & Lovejoy, W.S. (2016) Bargaining in supply chains. Management Science, 62(10), 3039–3058.
- Li, P., Castelo, N., Katona, Z. & Sarvary, M. (2024) Frontiers: determining the validity of large language models for automated perceptual analysis. Marketing Science, 43(2), 254–266.
- Loch, C.H. & Wu, Y. (2008) Social preferences and supply chain performance: an experimental study. Management Science, 54(11), 1835–1849.
- McManus, S. (2023) Can AI cut humans out of contract negotiations? BBC News. Available at: https://www.bbc.com/news/business-67238386.
- Meng, J. (2024) AI emerges as the frontier in behavioral science. Proceedings of the National Academy of Sciences, 121(10), e2401336121.
- Moghimifar, F., Li, Y.F., Thomson, R. & Haffari, G. (2024) Modelling political coalition negotiations using LLM-based agents. arXiv preprint arXiv:2402.11712.
- Nair, A., Narasimhan, R. & Bendoly, E. (2011) Coopetitive buyer–supplier relationship: an investigation of bargaining power, relational context, and investment strategies. Decision Sciences, 42(1), 93–127.
- Park, P.S., Goldstein, S., O'Gara, A., Chen, M. & Hendrycks, D. (2024) AI deception: a survey of examples, risks, and potential solutions. Patterns, 5(5).
- Rahwan, I., Cebrian, M., Obradovich, N., Bongard, J., Bonnefon, J.F., Breazeal, C. et al. (2019) Machine behaviour. Nature, 568(7753), 477–486.
- Spreitzenbarth, J.M., Bode, C. & Stuckenschmidt, H. (2024) Artificial intelligence and machine learning in purchasing and supply management: a mixed-methods review of the state-of-the-art in literature and practice. Journal of Purchasing and Supply Management, 30.
- Su, J., Lang, Y. & Chen, K.Y. (2023) Can AI solve newsvendor problem without making biased decisions? A behavioral experimental study. Available at SSRN: https://ssrn.com/abstract=4567157.
- Van Hoek, R., DeWitt, M., Lacity, M. & Johnson, T. (2022) How Walmart automated supplier negotiations. Harvard Business Review. Available at: https://hbr.org/2022/11/how-walmart-automated-supplier-negotiations [Accessed 27th July 2025].
- Wamba, S.F., Queiroz, M.M., Jabbour, C. J.C. & Shi, C.V. (2023) Are both generative AI and ChatGPT game changers for 21st-century operations and supply chain excellence? International Journal of Production Economics, 265, 109015.
- Westermann, H., Savelka, J. & Benyekhlef, K. (2023) LLMediator: GPT-4 assisted online dispute resolution. arXiv preprint arXiv:2307.16732.
- Wu, J.X., Wu, Y., Chen, K.Y. & Hua, L. (2023) Building socially intelligent AI systems: evidence from the trust game using artificial agents with deep learning. Management Science, 69(12), 7236–7252.
- Zao-Sanders, M. (2024) How people are really using GenAI. Harvard Business Review. Available at: https://hbr.org/2024/03/how-people-are-really-using-genai [Accessed: 6th May 2024].