See More

%PDF-1.7 %¿÷¢þ 1 0 obj << /Metadata 3 0 R /Names 4 0 R /OpenAction 5 0 R /Outlines 6 0 R /PageMode /UseOutlines /Pages 7 0 R /Type /Catalog >> endobj 2 0 obj << /Author (Yao Dou; Michel Galley; Baolin Peng; Chris Kedzie; Weixin Cai; Alan Ritter; Chris Quirk; Wei Xu; Jianfeng Gao) /Creator (arXiv GenPDF \(tex2pdf:e76afa9\)) /DOI (https://doi.org/10.48550/arXiv.2510.05444) /License (http://creativecommons.org/licenses/by/4.0/) /PTEX.Fullbanner (This is pdfTeX, Version 3.141592653-2.6-1.40.28 \(TeX Live 2025\) kpathsea version 6.4.1) /Producer (pikepdf 8.15.1) /Title (SimulatorArena: Are User Simulators Reliable Proxies for Multi-Turn Evaluation of AI Assistants?) /Trapped /False /arXivID (https://arxiv.org/abs/2510.05444v2) >> endobj 3 0 obj << /Subtype /XML /Type /Metadata /Length 1812 >> stream <alt><li xml:lang="x-default">SimulatorArena: Are User Simulators Reliable Proxies for Multi-Turn Evaluation of AI Assistants?</li></alt>

  • Yao Dou
  • Michel Galley
  • Baolin Peng
  • Chris Kedzie
  • Weixin Cai
  • Alan Ritter
  • Chris Quirk
  • Wei Xu
  • Jianfeng Gao
  • http://creativecommons.org/licenses/by/4.0/
  • cs.CL
    •  endstream endobj 4 0 obj << /Dests 8 0 R >> endobj 5 0 obj << /D [ 9 0 R /Fit ] /S /GoTo >> endobj 6 0 obj << /Count 15 /First 10 0 R /Last 11 0 R /Type /Outlines >> endobj 7 0 obj << /Count 79 /Kids [ 12 0 R 13 0 R 14 0 R ] /Type /Pages >> endobj 8 0 obj << /Kids [ 15 0 R 16 0 R ] /Limits [ (Doc-Start) (table.caption.8) ] >> endobj 9 0 obj << /Annots [ 17 0 R 18 0 R 19 0 R 20 0 R 21 0 R 22 0 R 23 0 R 24 0 R 25 0 R 26 0 R 27 0 R 28 0 R 29 0 R 30 0 R 31 0 R 32 0 R 33 0 R 34 0 R 35 0 R 36 0 R ] /Contents [ 37 0 R 38 0 R 39 0 R 40 0 R ] /Group 41 0 R /MediaBox [ 0 0 595.276 841.89 ] /Parent 42 0 R /Resources 43 0 R /Type /Page >> endobj 10 0 obj << /A 44 0 R /Next 45 0 R /Parent 6 0 R /Title 46 0 R >> endobj 11 0 obj << /A 47 0 R /Count -3 /First 48 0 R /Last 49 0 R /Parent 6 0 R /Prev 50 0 R /Title 51 0 R >> endobj 12 0 obj << /Count 36 /Kids [ 42 0 R 52 0 R 53 0 R 54 0 R 55 0 R 56 0 R ] /Parent 7 0 R /Type /Pages >> endobj 13 0 obj << /Count 36 /Kids [ 57 0 R 58 0 R 59 0 R 60 0 R 61 0 R 62 0 R ] /Parent 7 0 R /Type /Pages >> endobj 14 0 obj << /Count 7 /Kids [ 63 0 R 64 0 R ] /Parent 7 0 R /Type /Pages >> endobj 15 0 obj << /Kids [ 65 0 R 66 0 R 67 0 R 68 0 R 69 0 R 70 0 R ] /Limits [ (Doc-Start) (page.70) ] >> endobj 16 0 obj << /Kids [ 71 0 R 72 0 R ] /Limits [ (page.71) (table.caption.8) ] >> endobj 17 0 obj << /A << /S /URI /Type /Action /URI (mailto:[email protected]) >> /Border [ 0 0 0 ] /C [ 0 1 1 ] /H /I /Rect [ 122.163 642.72 213.819 659.167 ] /Subtype /Link /Type /Annot >> endobj 18 0 obj << /A << /S /URI /Type /Action /URI (mailto:[email protected]) >> /Border [ 0 0 0 ] /C [ 0 1 1 ] /H /I /Rect [ 217.804 642.72 345.326 659.167 ] /Subtype /Link /Type /Annot >> endobj 19 0 obj << /A << /S /URI /Type /Action /URI (https://aka.ms/SimulatorArena) >> /Border [ 0 0 0 ] /C [ 0 1 1 ] /H /I /Rect [ 374.831 642.72 490.05 659.167 ] /Subtype /Link /Type /Annot >> endobj 20 0 obj << /A << /D (cite.achiam2023gpt) /S /GoTo >> /Border [ 0 0 0 ] /C [ 0 1 0 ] /H /I /Rect [ 224.052 219.667 288.712 231.418 ] /Subtype /Link /Type /Annot >> endobj 21 0 obj << /A << /D (cite.achiam2023gpt) /S /GoTo >> /Border [ 0 0 0 ] /C [ 0 1 0 ] /H /I /Rect [ 69.87 206.118 94.03 217.868 ] /Subtype /Link /Type /Annot >> endobj 22 0 obj << /A << /D (cite.wang2024tutor) /S /GoTo >> /Border [ 0 0 0 ] /C [ 0 1 0 ] /H /I /Rect [ 235.987 179.019 288.712 190.77 ] /Subtype /Link /Type /Annot >> endobj 23 0 obj << /A << /D (cite.wang2024tutor) /S /GoTo >> /Border [ 0 0 0 ] /C [ 0 1 0 ] /H /I /Rect [ 69.87 165.879 94.117 177.221 ] /Subtype /Link /Type /Annot >> endobj 24 0 obj << /A << /D (cite.dhillon2024shaping) /S /GoTo >> /Border [ 0 0 0 ] /C [ 0 1 0 ] /H /I /Rect [ 223.141 165.879 288.712 177.221 ] /Subtype /Link /Type /Annot >> endobj 25 0 obj << /A << /D (cite.dhillon2024shaping) /S /GoTo >> /Border [ 0 0 0 ] /C [ 0 1 0 ] /H /I /Rect [ 69.87 151.921 93.877 163.672 ] /Subtype /Link /Type /Annot >> endobj 26 0 obj << /A << /D (cite.alexa) /S /GoTo >> /Border [ 0 0 0 ] /C [ 0 1 0 ] /H /I /Rect [ 204.454 97.724 259.846 109.475 ] /Subtype /Link /Type /Annot >> endobj 27 0 obj << /A << /D (cite.alexa) /S /GoTo >> /Border [ 0 0 0 ] /C [ 0 1 0 ] /H /I /Rect [ 263.695 97.724 287.942 109.475 ] /Subtype /Link /Type /Annot >> endobj 28 0 obj << /A << /D (cite.dstc) /S /GoTo >> /Border [ 0 0 0 ] /C [ 0 1 0 ] /H /I /Rect [ 69.87 84.584 113.43 95.926 ] /Subtype /Link /Type /Annot >> endobj 29 0 obj << /A << /D (cite.dstc) /S /GoTo >> /Border [ 0 0 0 ] /C [ 0 1 0 ] /H /I /Rect [ 116.335 84.584 139.709 95.926 ] /Subtype /Link /Type /Annot >> endobj 30 0 obj << /A << /D (cite.ji2022achieving) /S /GoTo >> /Border [ 0 0 0 ] /C [ 0 1 0 ] /H /I /Rect [ 142.913 84.584 174.596 95.926 ] /Subtype /Link /Type /Annot >> endobj 31 0 obj << /A << /D (cite.ji2022achieving) /S /GoTo >> /Border [ 0 0 0 ] /C [ 0 1 0 ] /H /I /Rect [ 177.501 84.584 200.875 95.926 ] /Subtype /Link /Type /Annot >> endobj 32 0 obj << /A << /D (cite.collins2024evaluating) /S /GoTo >> /Border [ 0 0 0 ] /C [ 0 1 0 ] /H /I /Rect [ 204.079 84.584 259.527 95.926 ] /Subtype /Link /Type /Annot >> endobj 33 0 obj << /A << /D (cite.collins2024evaluating) /S /GoTo >> /Border [ 0 0 0 ] /C [ 0 1 0 ] /H /I /Rect [ 262.432 84.584 285.806 95.926 ] /Subtype /Link /Type /Annot >> endobj 34 0 obj << /A << /D (cite.jurenka2024towards) /S /GoTo >> /Border [ 0 0 0 ] /C [ 0 1 0 ] /H /I /Rect [ 369.225 201.599 428.918 212.94 ] /Subtype /Link /Type /Annot >> endobj 35 0 obj << /A << /D (cite.jurenka2024towards) /S /GoTo >> /Border [ 0 0 0 ] /C [ 0 1 0 ] /H /I /Rect [ 432.37 201.599 456.115 212.94 ] /Subtype /Link /Type /Annot >> endobj 36 0 obj << /A << /S /URI /URI (https://arxiv.org/abs/2510.05444v2) >> /BS << /W 0 >> /NM (fitz-L0) /Rect [ 12 253.185 32 588.705 ] /Subtype /Link >> endobj 37 0 obj << /Length 10 /Filter /FlateDecode >> stream xœ+ä î | endstream endobj 38 0 obj << /Filter /FlateDecode /Length 3514 >> stream xÚ­ZÛ’Û6}÷W¨jBUY¼ì>l'ñÆIv{RÙ”ŽÄ‘¸¦È /Ï~ýžFÉ‘|ËÖÔA@£}9ÝP¸Ú¯ÂÕÓGáâùÕգϿ•©ZÉHè(V««›U*E–ÊUb”:[]íV/ƒåq¨ò¾i¿l×2 Š:ÿëz£e¸÷õF™0ø¥+Znã;~^Te~]¹q?ÙIÍÛ²pŸoÖʍ›ûãPõåæj©`À@Ôë߯ž­¤ÊD&«4"‹x[ß¼Ycy5ä}ÙÔ<½¹áç—ß¹gו]Ÿ×}÷wZDZ±ìndh„I³ÕF¥"”1¯üÛ:5AÞð_7O5É*YJši‘b8§Sž$ϐ™ˆ“xµ™ü±ÜŠŠšWUqfýD ™éúêC×ÿ*oªÒÉ㧵 q`û3b)”ü@ÉŒÀ“C[º³ûžÎ®Øý·,ÎQˆEÅHœ†½ÜÄ*x¥Âø,} i²ý_×1ˆ–o=“OòòucãÙûÜhmD Yod*TêTàË*w«?/ûš}†@*d¨,ìCU { ÁŸ‡²}}NxJ$I<]]}èêN>¼þ¿Ïi¯†²ãP?eëÏʼ¾!rúÛœÛ;–O£{‡‘AfI¬!jÖzN:“SÒ‘H´šûÓ¢±®c_æLü»ºëË~苹í_­èüöP7U³?g[2É„qì/TãáôÒvÛ¦knzŸÅ3MR84 ?JÎ$‚jØ)¡¡ð¹øfÕ’ëåæó§p¿Y2]áå&’I°k†û/öyD±Ü|öÙöø2xüQHBǽu;_=7bÛd²t>OÒDXiÈãDbD(¢Tiƒ)DvÙ÷üÜvÁaä¯sqì>Ÿ•Ù–Ϫi¤0ª©c‘â,Ùh¯»¾Í·ýÙ•x+™Èb[-8!•\¥ Öˆ`ЙHChìký³²A¹tpû!÷oÇfWT·_…&üá‡;<%÷ä­f‚²Þ¶EÞm8„=¤—ài"¦WÖûê GÁÐ;jéÀz8ô”5\+Éݾ¡°Xð‡üö¶*·6 v©+ òÚM> ÇäñR&|´ŽŽ</®¢iLE[ó²îø¥?ÜØ7ÕŽ÷@ñu—·;î¿iZOUÊY£¼5“Í»®@p¶Ž$JiÝ’ @”·E‹E°­mÁ߬ð