Azure OpenAI Service On Your Data ã®ä»çµã¿ãšäœ¿ãäžã§æ°ãä»ããã¹ããã€ã³ã
ã¯ããã«
Microsoft Build 2023 ã§çºè¡šããã Azure OpenAI Service ã® On Your Data ã®ãããªãã¯ãã¬ãã¥ãŒãéå§ ããŸãããäœæçã«ã¯ On Your Data ã¯æ¥æ¬åœå ã®å š Azure OpenAI Service ãŠãŒã¶ãŒã®ãã¡ 8 ïœ 9 å²çšåºŠã®ãŠãŒã¶ãŒãåŸ ã¡æãã§ããæ©èœã§ã¯ãªãããšæããŸãã(ryohtaka 調ã¹)
- What's new in Azure OpenAI Service - June 2023
- New easy way to add your data to Azure OpenAI Service (YouTube)
ããããå®éã« On Your Data ãæŽ»çšããããã«ã¯æ°ãä»ããã¹ããã€ã³ããæ°å€ãååšããŠãããæ£ç¢ºãªæåŸ å€ãæã£ãããã§äœ¿ãããšãéåžžã«éèŠã«ãªã£ãŠããŸããããã§ãæ¬èšäºã§ã¯ On Your Data ã®ä»çµã¿ãšäœ¿ãäžã§æ°ãä»ããã¹ããã€ã³ãã«çç®ããŠæ å ±ããŸãšããŸããã
ãªããããŒã¿ã®äžæºåãé€ã On Your Data ã®äœ¿ãæ¹èªäœã¯ãšãŠãç°¡åã§ããã¯ã€ãã¯ã¹ã¿ãŒããåããããã解説ããŠãããŠããèšäºãååšããŠããŸãã®ã§ããããåç §ããŠãã ããã
- ã¯ã€ã㯠ã¹ã¿ãŒã: ç¬èªã®ããŒã¿ã䜿çšã㊠Azure OpenAI ã¢ãã«ãšãã£ãããã
- Azure OpenAIã§ç¬èªããŒã¿è¿œå æ©èœ(Add your data)ã詊ããŠã¿ã
- ç¬èªãã¬ããžãããŒã³ãŒãã§ChatGPTã«é£æºïŒAzure Open AIãAdd your dataã
ã远èšã2023 幎 8 æã®æŽæ°ã§ããã€ãã®æ©èœã远å ãããŸãããOn Your Data ã¯ä»åŸãæ©èœã远å ãããŠããå¯èœæ§ããããŸãã®ã§ãå¿ ãããã¥ã¡ã³ãã®ææ°æ å ±ãåãããŠã確èªãã ããã
1. On Your Data ã§äœãå€ããïŒ
äœããã¬ã€ã¯ã¹ã«ãŒãªã®ãããŸãšããŸãã
1.1. ç°¡æçãªã¢ããªã±ãŒã·ã§ã³ã®ãããã€
Azure OpenAI Service On Your Data ã䜿ããšãNew Bing ã®ãã㪠UX ã®ãã£ããæ€çŽ¢ããŠãŒã¶ãŒç¬èªã®ããŒã¿ã察象ã«ããŠè¡ãç°¡æç㪠Web ã¢ããªã±ãŒã·ã§ã³ãã³ãŒããæžããã«éçºããããšãã§ããŸããäžå³ã®ç Žç·ã§å²ã£ãéšåã§ãã

ãããŸã§ãããããã Retrieval Augmented Generation (RAG) ãšãããã¶ã€ã³ãã¿ãŒã³ã«åãã°ãŠãŒã¶ãŒç¬èªã®ããŒã¿ã«å¯Ÿãããã£ããæ€çŽ¢ãå®çŸããããšã¯ã§ããŸããããããè€éãªåŠçãå®è£ ããå¿ èŠããããŸããããã®éçºãçååããŠãããç¹ã On Your Data ã®ãã¬ã€ã¯ã¹ã«ãŒã®ã²ãšã€ã§ãã
ãªããæçµçã«äœãããç°¡æã¢ããªã±ãŒã·ã§ã³ã® UX ã¯ä»¥åããååšããŠããå ¬åŒã®ãµã³ãã«å®è£ ãšã»ãŒåãã§ãã
åè
- microsoft/sample-app-aoai-chatGPT (On Your Data ã§ãããã€ããã Web ã¢ããªã±ãŒã·ã§ã³ã®ãœãŒã¹ã³ãŒã)
- Azure-Samples/azure-search-openai-demo (以åããå ¬éãããŠããå ¬åŒãµã³ãã«å®è£ )
- æ¥åããŒã¿ã掻çšããAIãã£ããã·ã¹ãã ã¯ãŒã¯ã·ã§ãã (äžèšãµã³ãã«å®è£ ã step-by-step ã«åå²ããŠæ¥æ¬èªã§ãããããã解説ããŠãããŠããã³ã³ãã³ã)
[è£è¶³] Retrieval Augmented Generation (RAG) ãšã¯ïŒ
æ¢ã«åºãèªèããå§ããŠããŸãããçæç³»ã®å€§èŠæš¡èšèªã¢ãã« (LLM) ã¯ãããŸã§ã Web äžã®èšå€§ãªããã¹ãããŒã¿ãããšã«åŠç¿ãè¡ããæ¬¡ã«æ¥ãå¯èœæ§ã®é«ãåèªã®äºæž¬ãç¹°ãè¿ãããšã§ãããããããã¹ããçæããŸãããã®ããã(äºå®ãšããŠ) æ£ãããªãæ å ±ãåºåããããã«ã·ããŒã·ã§ã³ãèµ·ããåŸãããšã«çæããå¿ èŠããããLLM ã«å¯ŸããŠçŽæ¥ç¥èãåãã®ã¯ããŸãé©ããäœ¿ãæ¹ã§ã¯ãããŸããã
ããããäžæ¹ã§ LLM ãæ å ±æ€çŽ¢ã«å¿çšããããšããã¢ãããŒã·ã§ã³ãå°ãªãããååšããŠããããšãäºå®ã§ãæè¿ã¯ãã®è§£æ±ºçãšã㊠Retrieval Augmented Generation (RAG) ãšåŒã°ãããã¶ã€ã³ãã¿ãŒã³ãäžè¬çã«ãªãã€ã€ãããŸããRAG ã¯äžèšã§è¡šçŸãããšãã¬ããžããŒã¹ã®å€éšåã§ãRAG ã§ã¯ãŠãŒã¶ãŒããã®åãã«å¯ŸããŠãããã¯ãšã³ãã«çœ®ãããã¬ããžããŒã¹ (e.g. Azure Cognitive Search) ã§æ€çŽ¢ãè¡ãããã®çµæãèžãŸããŠãŠãŒã¶ãŒã®ããã®åãããã«å¯ŸããåçãçæããŸãã
ã¡ãªã¿ã«ããã®ããã« LLM ã«æ ¹æ ã®æ å ± (e.g. ç¬èªããŒã¿ã«å¯Ÿããæ€çŽ¢ãWeb æ€çŽ¢ãç) ã«åºã¥ãããŠåçãè¡ãããšãã°ã©ãŠã³ãã£ã³ã° (Grounding) ãšåŒã³ãŸãã
RAG ã¢ããªã±ãŒã·ã§ã³ã®ã€ã¡ãŒãž

- RAG ã¢ããªã¯èšèªã¢ãã«ã䜿ã£ãŠãŠãŒã¶ãŒããã®èªç¶èšèªã«ããåãããããã¬ããžããŒã¹ã®æ€çŽ¢ããŒã¯ãŒããžå€æããã
- RAG ã¢ããªã¯æ€çŽ¢ããŒã¯ãŒãã䜿ã£ãŠãã¬ããžããŒã¹ã«å¯ŸããŠæ€çŽ¢ãè¡ãã
- ãã¬ããžããŒã¹ã¯äºåã«äœæããŠãããæ€çŽ¢ã€ã³ããã¯ã¹ã«ãã£ãŠæ€çŽ¢çµæãè¿ãã
- RAG ã¢ããªã¯ãã¬ããžããŒã¹ã®æ€çŽ¢çµæ (åŒçšæ å ±) ãåèã«ããŠäœ¿ã£ãŠãŠãŒã¶ãŒãžã®åçæãçæããã
ãŠãŒã¶ãŒããã®ãªã¯ãšã¹ãã®äŸ (ã¡ãã»ãŒãžéšåã®ã¿)
"messages": [
{
"role": "system",
"content": "You are an AI assistant that helps people find information. You retrieve English documents, and you should read them carefully in English and answer in English."
},
{
"role": "user",
"content": "What is Microsoft Azrue?"
}
]
RAG ã¢ããªããã®ã¬ã¹ãã³ã¹ã®äŸ (ã¡ãã»ãŒãžéšåã®ã¿)
"messages": [
{
"index": 0,
"role": "tool",
"content": "{\"citations\": [{\"content\": \"The Azure Resource Manager, introduced in 2014, enables users to create groups of related services so that closely coupled resources can be deployed, managed, and monitored together.\\n_START_SECTION_\\nPrivacy\\n_START_PARAGRAPH_\\nMicrosoft has stated that, per the USA Patriot Act, the US government could have access to the data even if the hosted company is not American and the data resides outside the USA. However, Microsoft Azure is compliant with the E.U. Data Protection Directive (95/46/EC). To manage privacy and security-related concerns, Microsoft has created a Microsoft Azure Trust Center, and Microsoft Azure has several of its services compliant with several compliance programs including ISO 27001:2005 and HIPAA. A full and current listing can be found on the Microsoft Azure Trust Center Compliance page. Of special note, Microsoft Azure has been granted JAB Provisional Authority to Operate (P-ATO) from the U.S. government in accordance with guidelines spelled out under the Federal Risk and Authorization Management Program (FedRAMP), a U.S. government program that provides a standardized approach to security assessment, authorization, and continuous monitoring for cloud services used by the federal government.\\n_START_SECTION_\\nMeek-Azure\\n_START_PARAGRAPH_\\nThe Tor Project uses Microsoft Azure as a domain front in its obfuscation plugin, Meek. Meek-Azure is commonly used in China where Tor relays are regularly blocked and disabled. Meek-Azure became the last Meek after the banning of Amazon cloud services.\", \"id\": null, \"title\": \"_START_ARTICLE_\", \"filepath\": \"Q725967_Microsoft Azure.txt\", \"url\": \"https://xxx.blob.core.windows.net/wiki40ben-ms-articles/Q725967_Microsoft%20Azure.txt\", \"metadata\": {\"chunking\": \"orignal document size=330. Scores=6.4695673 and None.Org Highlight count=21.\"}, \"chunk_id\": \"0\"}, {\"content\": \"_START_ARTICLE_\\nMicrosoft Azure SQL Database\\n_START_SECTION_\\nDesign\\n_START_PARAGRAPH_\\nAzure SQL Database shares the SQL Server 2016 codebase. It is compatible with SQL Server 2014 and 2016 and most of the features available in SQL Server 2016 are available in Azure SQL Database. A list of incompatibilities is published by Microsoft.\", \"id\": null, \"title\": \"_START_ARTICLE_\", \"filepath\": \"Q1187310_Microsoft Azure SQL Database.txt\", \"url\": \"https://xxx.blob.core.windows.net/wiki40ben-ms-articles/Q1187310_Microsoft%20Azure%20SQL%20Database.txt\", \"metadata\": {\"chunking\": \"orignal document size=82. Scores=6.164525 and None.Org Highlight count=7.\"}, \"chunk_id\": \"0\"}, {\"content\": \"_START_ARTICLE_\\nMicrosoft Azure Web Sites\\n_START_SECTION_\\nImplementation\\n_START_PARAGRAPH_\\nMicrosoft Azure Web Sites is implemented as websites that are dynamically created on-demand on servers running Windows Server 2012 and IIS 8.0. When a client posts a request to a web site, Microsoft Azure Web Sites dynamically provisions the site on one of the Azure virtual machines pointing it at content stored in Azure Storage containers. The Azure Virtual Machines are deployed in groups called \\\"Stamps\\\", which may contain hundreds of such machines. Microsoft deploys these stamps in its Azure data centers across the world, and adds more stamps as demand grows.\\n_START_SECTION_\\nTiers\\n_START_PARAGRAPH_\\nAzure Web Sites services are offered in 4 tiers. The entry-level tier is the \\\"free\\\" tier. The free tier supports up to 10 websites with 1 GB of content storage, and is limited to 165 MB of daily data egress. The first for-pay tier is the \\\"Shared\\\" tier. Shared tier sites support custom domains and can be scaled out to up to 6 instances. The current highest for-pay tier is the \\\"Standard\\\" tier. Standard tier websites run on VMs dedicated exclusively to a single customer's websites. The Standard tier supports SSL (both SNI and IP-based), scaling out to up to 10 instances, and file storage of up to 50 GB of content._NEWLINE_Basic and Standard tier websites can be deployed on 3 sizes of virtual machines: Small VMs with 1 virtual CPU and 1.75GB of RAM, Medium VMs with 2 virtual CPUs and 3.5 GB of RAM, and Large VMs with 4 virtual CPUs and 7GB of available RAM.\\n_START_SECTION_\\nHistory\\n_START_PARAGRAPH_\\nMicrosoft initially offered a basic web hosting service as part of Office Live Small Business, which was launched in late 2007. Office Live Small Business offered customers free and commercial web hosting with a built-in system for creating websites based on build-in templates and a site creation wizard._NEWLINE_When Microsoft started allocating resources into developing its numerous cloud solutions, a group was formed in Microsoft Azure to develop Microsoft Azure Web Sites. Microsoft Azure Web Sites was announced in June 2012 as a preview release._NEWLINE_In parallel, Microsoft developed Microsoft Azure Pack, which offers the same technology that can be installed as a private-cloud on sets of servers at a customer's site and under direct customer control._NEWLINE_In mid-2013, both Microsoft Azure Web Sites and Microsoft Azure Pack were officially released to the public.\\n_START_SECTION_\\nDatacenters\\n_START_PARAGRAPH_\\nMicrosoft operates Microsoft Azure in a number of data centers throughout the world with Microsoft Azure Web Sites available in a subset of these. When deploying websites on Microsoft Azure Web Sites customers can choose to deploy sites in a data center in any of the following geographical regions: East US, North central US, West US, East Asia, Brazil South, North Europe, West Europe, Japan East, Japan West or East Asia.\", \"id\": null, \"title\": \"_START_ARTICLE_\", \"filepath\": \"Q17048728_Microsoft Azure Web Sites.txt\", \"url\": \"https://xxx.blob.core.windows.net/wiki40ben-ms-articles/Q17048728_Microsoft%20Azure%20Web%20Sites.txt\", \"metadata\": {\"chunking\": \"orignal document size=660. Scores=5.822534 and None.Org Highlight count=39.\"}, \"chunk_id\": \"0\"}, {\"content\": \"_START_ARTICLE_\\nMicrosoft Azure Dev Tools for Teaching\\n_START_SECTION_\\nHistory\\n_START_PARAGRAPH_\\nAll Student get access to Cloud resources and Azure credit student must register at the Microsoft Azure for Student and verify their identity through their verified educational institutions. If an institution is not listed on the available list, the user may manually verify their student status by uploading a proof such as an ID card._NEWLINE_The Microsoft student developer tools programme was announced by Bill Gates as DreamSpark on February 20, 2008 during a speech at Stanford University. It is estimated that up to 35 million students will be able to access these software titles through this program. The service was renamed to Microsoft Imagine on September 7, 2016, to better align with the annual Imagine Cup competition hosted by Microsoft. and renamed to Azure Dev Tools for Teaching and Azure for Students in June 2018\\n_START_SECTION_\\nRepacked as Azure Dev Tools for Teaching\\n_START_PARAGRAPH_\\nThe Microsoft Azure Dev Tools for Teaching on 13 February 2019 with Cloud services for students using the Microsoft Azure cloud and incorporating Microsoft Learn online learning resources.\\n_START_SECTION_\\nVerification\\n_START_PARAGRAPH_\\nProof of student status is required to download software and obtain product keys. Students can verify their identity using ISIC cards, access codes ordered by school administrators, or .edu academic email addresses. Students remain verified for 12 months afterwards and can renew after the 12 month period using their academic email. If students can't find their school, they can manually submit a response with a proof of student status.\\n_START_SECTION_\\nProducts offered\\n_START_PARAGRAPH_\\nSeveral development software titles are available for download through the program. They include:\", \"id\": null, \"title\": \"_START_ARTICLE_\", \"filepath\": \"Q909738_Microsoft Azure Dev Tools for Teaching.txt\", \"url\": \"https://xxx.blob.core.windows.net/wiki40ben-ms-articles/Q909738_Microsoft%20Azure%20Dev%20Tools%20for%20Teaching.txt\", \"metadata\": {\"chunking\": \"orignal document size=380. Scores=5.5316896 and None.Org Highlight count=19.\"}, \"chunk_id\": \"0\"}, {\"content\": \"_START_ARTICLE_\\nMicrosoft Azure\\n_START_SECTION_\\nServices\\n_START_PARAGRAPH_\\nMicrosoft lists over 600 Azure services, of which some are covered below:\\n_START_SECTION_\\nMedia services\\n_START_PARAGRAPH_\\nA PaaS offering that can be used for encoding, content protection, streaming, or analytics.\\n_START_SECTION_\\nCDN\\n_START_PARAGRAPH_\\nA global content delivery network (CDN) for audio, video, applications, images, and other static files. It can be used to cache static assets of websites geographically closer to users to increase performance. The network can be managed by a REST based HTTP API._NEWLINE_Azure has 54 point of presence locations worldwide (also known as Edge locations) as of August 2018.\\n_START_SECTION_\\nAzure Blockchain Workbench\\n_START_PARAGRAPH_\\nThrough Azure Blockchain Workbench, Microsoft is providing the required infrastructure to setup a consortium network in multiple topologies using a variety of consensus mechanisms. Microsoft provides integration from these blockchain platforms to other Microsoft services to streamline development of distributed applications. Microsoft supports many general-purpose blockchains including Ethereum and Hyperledger Fabric and purpose-built blockchains like Corda.\\n_START_SECTION_\\nFunctions\\n_START_PARAGRAPH_\\nAzure functions are used in serverless computing architectures where subscribers can execute code as a Function-as-a-Service (FaaS) without managing the underlying server resources.\\n_START_SECTION_\\nRegional expansion and examples\\n_START_PARAGRAPH_\\nAzure is generally available in 54 regions around the world. Microsoft has announced an additional 12 regions to be opened soon (as of October 2018). Microsoft is the first hyper-scale cloud provider that has committed to building facilities on the continent of Africa with two regions located in South Africa. An Azure geography contains multiple Azure Regions, such as example \\u201cNorth Europe\\u201d (Dublin, Ireland), \\u201cWest Europe\\u201d (Amsterdam, Netherlands). Where a location represents the city or area of the Azure Region. Each Azure Region is paired with another region within the same geography; this makes them a regional pair. In this example, Amsterdam and Dublin are the locations which form the regional-pair._NEWLINE_Microsoft has some Gold partners available across the globe to sell its products. In August 2018, Toyota Tsusho began a partnership with Microsoft to create fish farming tools using the Microsoft Azure application suite for IoT technologies related to water management. Developed in part by researchers from Kindai University, the water pump mechanisms use artificial intelligence to count the number of fish on a conveyor belt, analyze the number of fish, and deduce the effectiveness of water flow from the data the fish provide. The specific computer programs used in the process fall under the Azure Machine Learning and the Azure IoT Hub platforms.\\n_START_SECTION_\\nDesign\\n_START_PARAGRAPH_\\nMicrosoft Azure uses a specialized operating system, called Microsoft Azure, to run its \\\"fabric layer\\\": A cluster hosted at Microsoft's data centers that manages computing and storage resources of the computers and provisions the resources (or a subset of them) to applications running on top of Microsoft Azure. Microsoft Azure has been described as a \\\"cloud layer\\\" on top of a number of Windows Server systems, which use Windows Server 2008 and a customized version of Hyper-V, known as the Microsoft Azure Hypervisor to provide virtualization of services._NEWLINE_Scaling and reliability are controlled by the Microsoft Azure Fabric Controller, which ensures the services and environment do not fail if one or more of the servers fails within the Microsoft data center, and which also provides the management of the user's Web application such as memory allocation and load balancing._NEWLINE_Azure provides an API built on REST, HTTP, and XML that allows a developer to interact with the services provided by Microsoft Azure. Microsoft also provides a client-side managed class library that encapsulates the functions of interacting with the services. It also integrates with Microsoft Visual Studio, Git, and Eclipse._NEWLINE_In addition to interacting with services via API, users can manage Azure services using the Web-based Azure Portal, which reached General Availability in December 2015. The portal allows users to browse active resources, modify settings, launch new resources, and view basic monitoring data from active virtual machines and services.\\n_START_SECTION_\\nDeployment models\\n_START_PARAGRAPH_\\nMicrosoft Azure offers two deployment models for cloud resources: the \\\"classic\\\" deployment model and the Azure Resource Manager. In the classic model, each Azure resource (virtual machine, SQL database, etc.) was managed individually\", \"id\": null, \"title\": \"_START_ARTICLE_\", \"filepath\": \"Q725967_Microsoft Azure.txt\", \"url\": \"https://xxx.blob.core.windows.net/wiki40ben-ms-articles/Q725967_Microsoft%20Azure.txt\", \"metadata\": {\"chunking\": \"orignal document size=1005. Scores=5.380728 and None.Org Highlight count=50.\"}, \"chunk_id\": \"1\"}], \"intent\": \"What is Microsoft Azrue?\"}",
"end_turn": false
},
{
"index": 1,
"role": "assistant",
"content": "Microsoft Azure is a cloud computing platform developed by Microsoft that allows individuals and businesses to build, deploy, and manage applications and services in the cloud [doc1]. It provides a wide range of services and tools for building, deploying, and managing applications and services, including virtual machines, web applications, databases, storage, networking, analytics, and more [doc5].",
"end_turn": true
}
]
åè
- Retrieval Augmented Generation (RAG)
- Retrieval Augmented Generation: Streamlining the creation of intelligent natural language processing models
1.2. ã°ã©ãŠã³ãã£ã³ã°å¯Ÿè±¡ãã¬ããžããŒã¹ã®åãæ¿ã
REST API ã§ã®ãªã¯ãšã¹ãæã«ææ°ã®ãã¬ãã¥ãŒç API Version ã§ãã 2023-06-01-preview ãæå®ããããã§ dataSources ãã©ã¡ãŒã¿ãäžããããšã§ãã°ã©ãŠã³ãã£ã³ã°å¯Ÿè±¡ã®ãã¬ããžããŒã¹ãæå®ããããšãã§ããŸããããã«ããããªã¯ãšã¹ãã®ãã³ã«ãä»åã¯ãã£ã¡ã®ãã¬ããžããŒã¹ã§ã°ã©ãŠã³ãã£ã³ã°ããŠããšãã£ã颚ã«ãã¬ããžããŒã¹ãåãæ¿ããããšãã§ããããã«ãªããŸãã
å人çã«ã¯ãåè¿°ã®ç°¡æã¢ããªã±ãŒã·ã§ã³ã®éçºããã倧ããªãã¬ã€ã¯ã¹ã«ãŒã ãšæããŠããŸãã
ãã¬ããžããŒã¹åãæ¿ãã®ã€ã¡ãŒãž

ãªã¯ãšã¹ãã®ãã©ãŒããã
ãªã¯ãšã¹ãã®ãã©ãŒãããã¯ä»¥äžã®ããã«ãªããŸãããã®ããã«ããŠãã°ã©ãŠã³ãã£ã³ã°å¯Ÿè±¡ã®ãã¬ããžããŒã¹ (å ·äœçã«ã¯ Azure Cognitive Search ã€ã³ããã¯ã¹) ãæå®ããããšãã§ããŸãã
-
<your-aoai-resource-name>: Azrue OpenAI Service ã®ãªãœãŒã¹å -
<your-deployment-name>: 察象ã®ãããã€å (Chat Completions ãå¯èœãªã¢ãã«ã®ãã¡gpt-35-turboãããã¯gpt-4ãæå®ãã) -
<your-aoai-key>: Azure OpenAI Service ã®ã㌠-
<target-cognitive-search-resource-name>: ã°ã©ãŠã³ãã£ã³ã°å¯Ÿè±¡ã® Azure Cognitive Search ã®ãªãœãŒã¹å -
<target-cognitive-search-key>: ã°ã©ãŠã³ãã£ã³ã°å¯Ÿè±¡ã® Azure Cognitive Search ã®ã㌠-
<target-cognitive-search-index-name>: ã°ã©ãŠã³ãã£ã³ã°å¯Ÿè±¡ã® Azure Cognitive Search ã€ã³ããã¯ã¹å
POST https://<your-aoai-resource-name>/openai/deployments/<your-deployment-name>/extensions/chat/completions?api-version=2023-06-01-preview
Content-Type: application/json
api-key: <your-aoai-key>
chatgpt_url: https://<your-aoai-resource-name>/openai/deployments/<your-deployment-name>/chat/completions?api-version=2023-06-01-preview
chatgpt_key: <your-aoai-key>
{
"dataSources": [
{
"type": "AzureCognitiveSearch",
"parameters": {
"endpoint": "https://<target-cognitive-search-resource-name>.search.windows.net",
"key": "<target-cognitive-search-key>",
"indexName": "<target-cognitive-search-index-name>"
}
}
],
"messages": [
{
"role": "system",
"content": "You are an AI assistant that helps people find information. You retrieve English documents, and you should read them carefully in English and answer in English."
},
{
"role": "user",
"content": "What is Microsoft Azrue?"
}
]
}
åè
2. æ°ãä»ããã¹ããã€ã³ã
ãã以éã§ã¯æ°ãä»ããã¹ããã€ã³ãã«ã€ããŠãŸãšããŸãã
2.1. 察å¿ã¢ãã«
On Your Data ã¯ä»¥äžã®ã¢ãã«ã§å©çšå¯èœã§ãã
gpt-35-turbogpt-35-turbo-16kgpt-4gpt-4-32k
äŸãã° gpt-35-turbo ã®æå€§ããŒã¯ã³ã¯ 4,096 (æ¥æ¬èªã®å Žå çŽ 3,700 æåçšåºŠ) ã§ãã®ã§ãã·ã¹ãã ã¡ãã»ãŒãžããŠãŒã¶ãŒã®ããã³ããããã¬ããžããŒã¹ããã®æ€çŽ¢çµæããŠãŒã¶ãŒãžã®åç ãã¹ãŠããã®æå€§ããŒã¯ã³ã«åãŸããããªããã³ããèšèšãšããã¬ããžããŒã¹ã®æŽå (çããã£ã³ã¯ã«æçãªæ
å ±ãè©°ãŸã£ãŠããããŒã¿ã®æŽå) ãå¿
èŠã«ãªããŸãã
åè
- ç¬èªã®ããŒã¿ã«åºã¥ã Azure OpenAI (ãã¬ãã¥ãŒ)
- Azure OpenAI Service ã¢ãã«
- OpenAI èšèªã¢ãã«ã§æ¥æ¬èªãæ±ãéã®ããŒã¯ã³æ°æšå®ææš
- OpenAI èšèªã¢ãã«ããšã®ãšã³ã³ãŒãã£ã³ã°äžèЧ
2.2. æ¥æ¬èª
2.2.1. ã¢ãã©ã€ã¶ãŒã®èšèª
On Your Data ã«ã¯ 3 éãã®ãã¬ããžããŒã¹æå®æ¹æ³ããããŸãã
- æ¢åã® Azure Cognitive Search ã€ã³ããã¯ã¹ãæå® (ãŠãŒã¶ãŒä»»æã®ã€ã³ããã¯ã¹ãäœæ)
- ããŒã¿ãæ ŒçŽããã Azure Blob ã³ã³ãããŒãæå® (ã€ã³ããã¯ã¹èªåäœæ)
- Azure Blob ã³ã³ãããŒã«ãã¡ã€ã«ãã¢ããããŒã (ã€ã³ããã¯ã¹èªåäœæ)
ãã®ãããã®éžæè¢ãåã£ãå Žåããæçµçãªæ§æã¯ã»ãŒåãã«ãªããŸãããæ³šæç¹ãšããŠã¯ã2 ãš 3 ãéžæããå Žåã« Azure Cognitive Search ã®ã€ã³ããã¯ã¹ãèªåäœæãããã¢ãã©ã€ã¶ãŒã®èšèªãè±èªåºå®ã§ããç¹ã§ããæ€çŽ¢å¯Ÿè±¡ãè±èªã§ããã°ç¹ã«åé¡ãããŸããããæ¥æ¬èªã®ææžãæ€çŽ¢å¯Ÿè±¡ãšãããå Žåã¯ã€ã³ããã¯ã¹äœææã®åå¥è§£æãæ£ããè¡ããããç®çã®ææžãäžæãæ€çŽ¢ã§ããªããªã£ãŠããŸããŸãã
1. æ¢åã®ã€ã³ããã¯ã¹ãæå®

2. Blob ã¹ãã¬ãŒãžãæå® / 3. ãã¡ã€ã«ã¢ããããŒã

ãã£ãŠãå¿ ç¶çã«æ¥æ¬èªãæ€çŽ¢å¯Ÿè±¡ã«ãããå Žå㯠1 ãå¯äžã®éžæè¢ãšãªããŸãã Azure Cognitive Search ã®ã¯ã©ã€ã¢ã³ãã©ã€ãã©ãª ã API ã®æ±ãã«æ £ããŠããããã§ããã°æ¥æ¬èªã¢ãã©ã€ã¶ãŒãæå®ããã€ã³ããã¯ã¹ãèªåã§äœæããŸããããã§ãªãå Žåã¯ãè±èªä»¥å€ã®ã¢ãã©ã€ã¶ãŒã䜿ã£ãŠã€ã³ããã¯ã¹ãäœæããããã®ããŒã«ãå ¬åŒãµã³ãã«ãšããŠå ¬éãããŠããŸãã®ã§ãããã䜿ã£ãŠã€ã³ããã¯ã¹ãäœæããŸãããã®ããŒã«ã䜿ããšããã¬ããžããŒã¹ãšããŠã€ã³ããã¯ã¹ãæå®ããéã«æ±ãããããã£ãŒã«ã ãå šãŠäœæããŠãããŸãã
åè
- Azure Cognitive Search ã€ã³ããã¯ã¹ã®æååãã£ãŒã«ãã«èšèªã¢ãã©ã€ã¶ãŒã远å ãã
- Azure Cognitive Search ã§ã®é¢é£æ§ãšã¹ã³ã¢ãªã³ã°
- microsoft/sample-app-aoai-chatGPT (ã€ã³ããã¯ã¹äœæããŒã«)
- çæAIçšCognitive Searchã®èšèªã¢ãã©ã€ã¶ãŒãæ¥æ¬èªã«ããã (äžèšããŒã«ã®äœ¿ãæ¹ã®æ¥æ¬èªè§£èª¬èšäº)
2.2.2. ã·ã¹ãã ã¡ãã»ãŒãž
æ¥æ¬èªã®ãã¬ããžããŒã¹ãå¯Ÿè±¡ã«æ¥æ¬èªã®ããã³ããã§æ€çŽ¢ãè¡ãå Žåãçšã«åçãè±èª (OpenAI ã®èšèªã¢ãã«ã®ç¬¬äžèšèªïŒ) ã«ãªã£ãŠããŸãããšããããŸãããã®åé¡ã«å¯ŸããŠã¯ãäžèšã®ãããªã·ã¹ãã ã¡ãã»ãŒãžã远å ããŠæç¢ºã«æ¥æ¬èªã§åçããããæç€ºããããšã§äœæžã§ããŸãã
ããªãã¯ãæ å ±ã®æ€çŽ¢ãæ¯æŽãã AI ã¢ã·ã¹ã¿ã³ãã§ãã æ¥æ¬èªã®ããã¥ã¡ã³ããååŸãããããæ¥æ¬èªã§æ³šææ·±ãèªã¿ãæ¥æ¬èªã§åçããå¿ èŠããããŸãã

åè
2.2.3. Azure Cognitive Search ã®ã»ãã³ãã£ãã¯æ€çŽ¢
Azure Cognitive Search ã®ã»ãã³ãã£ãã¯æ€çŽ¢ã¯ãAzure Cognitive Search ã®æ€çŽ¢å質ãäžããããšãã§ãããã¬ãã¥ãŒäžã®æ©èœã§ãã察象èšèªãè±èªã®å Žå㯠On Your Data ãšå
±ã«äœ¿ãããšãã§ããŸãããå·çæç¹ã§è±èªä»¥å€ã®èšèªã«ã¯å¯Ÿå¿ããŠããªããããæ¥æ¬èªç°å¢ã§ã¯äœ¿çšã§ããªãããšã«æ³šæãå¿
èŠã§ãã

åè
- ç¬èªã®ããŒã¿ã«åºã¥ã Azure OpenAI (ãã¬ãã¥ãŒ) - ã»ãã³ãã£ãã¯æ€çŽ¢
- Azure Cognitive Search ã§ã®ã»ãã³ãã£ãã¯æ€çŽ¢
2.3. ã€ã³ããã¯ã¹äœæåŠçã®åå®è¡
åè¿°ã® 3 éãã®ãã¬ããžããŒã¹æå®æ¹æ³ã®ãã¡ãã€ã³ããã¯ã¹ãèªåäœæããã 2 ãš 3 ã§ã¯ ã€ã³ãã¯ãµãŒãäœæããããã€ã³ããã¯ã¹ã®åäœæãè¡ããŸããããã®ããããã¬ããžããŒã¹ãæŽæ°ãããŠãããããªç¶æ³ (éåžžå€ãã®ãŠãŒã¹ã±ãŒã¹ãããã«è©²åœãããšæããã) ã§ã¯ã1 ãå¯äžã®éžæè¢ã«ãªããšæãããŸãã
- æ¢åã® Azure Cognitive Search ã€ã³ããã¯ã¹ãæå® (ãŠãŒã¶ãŒä»»æã®ã€ã³ããã¯ã¹ãäœæ)
- ããŒã¿ãæ ŒçŽããã Azure Blob ã³ã³ãããŒãæå® (ã€ã³ããã¯ã¹èªåäœæ)
- Azure Blob ã³ã³ãããŒã«ãã¡ã€ã«ãã¢ããããŒã (ã€ã³ããã¯ã¹èªåäœæ)
ã€ã³ããã¯ã¹åäœæã®æ¹æ³ã¯äžèšã® 2 éããèããããŸãã
- ã€ã³ãã¯ãµãŒã®åå®è¡
- Azure Cognitive Search ã®ã¯ã©ã€ã¢ã³ãã©ã€ãã©ãªã§å®è£ ããåŠçã§åäœæ (ä»»æã®ãžã§ãã¹ã±ãžã¥ãŒã©ãŒã§å®è¡)
ã远èšã 2. ã«éãã2023 幎 8 æã®ã¢ããããŒãã§ã€ã³ããã¯ã¹ã®èªåæŽæ°æ©èœã远å ãããŸããã
åè
- Azure Cognitive Search ã®ã€ã³ãã¯ãµãŒ
- Azure Cognitive Search ã®ã€ã³ãã¯ãµãŒãã¹ã±ãžã¥ãŒã«ãã
2.4. ãã£ã³ãã³ã°ã®éèŠæ§
ãã£ããæ€çŽ¢ã«ãããŠè¯ãæ€çŽ¢çµæãåŸãããã«ã¯ãããã¥ã¡ã³ãã®è³ªã®é«ããã£ã³ã¯ (åå²) ãäœãããã©ãããèã«ãªããŸãã(äžå³)

åè¿°ã®ãšãããçŸæç¹ã§æ¥æ¬èªç°å¢ãæ³å®ããå Žåãæ¢åã® Azure Cognitive Search ã€ã³ããã¯ã¹ãæå®ããäºå®äžå¯äžã®éžæè¢ãšãªããŸãããã®éžæè¢ãåãå Žåã®ãã£ã³ãã³ã°ã®æ¹æ³ã»ã¿ã€ãã³ã°ã¯äžèš 2 éããèããããŸããã©ã¡ãã®éžæè¢ãåããã©ããã¯ããããžã§ã¯ãã®è²¬ä»»åçç¹ãåäœæã®é »åºŠãéçºè ã®ã奜ã¿çã«ããæ±ºããšæãããŸãã(ã±ãŒã¹ãã€ã±ãŒã¹)
- ä»»æã®ãã£ã³ãã³ã°åŠçãç¬ç«ããŠè¡ãããã£ã³ãã³ã°ããåŸã« Blob ã³ã³ãããŒã«æ ŒçŽããã®åŸãã€ã³ãã¯ãµãŒã«ããã€ã³ããã¯ã¹ãäœæã
- åæã«è¡ããLangChain çã®ã©ã€ãã©ãªã䜿ã£ãŠä»»æã®ãã£ã³ãã³ã°ãè¡ãã€ã€ Azure Cognitive Search ã®ã¯ã©ã€ã¢ã³ãã©ã€ãã©ãª ã«ãã£ãŠã€ã³ããã¯ã¹ãäœæã
ãªãããã£ã³ãã³ã°ã®ã¿ã«æ³šç®ããŠãããã£ãšäžèšã®ãããªãã©ã¡ãŒã¿ãŒãååšããŠããããããã¯å šãŠæ€çŽ¢ã¯ãªãªãã£ã®åäžãç®æãéã®è©Šè¡é¯èª€ã®èŠçŽ ãšãªããŸãã
- åçŽã«ã¶ã€åãããã®ã or ãªãŒããŒã©ãããããã®ã
- ãã£ã³ã¯ã®é·ãã¯ã©ãããã®ã
- ãªãŒããŒã©ããã®é·ãã¯ã©ãããã®ã
- ãã£ã³ã¯ã¯æåæ°ã§èŠãã®ã or ããŒã¯ã³æ°ã§èŠãã®ã
- etc
åè
2.5. Private Endpoint ã®ãµããŒã
æ¢åã® Azure Cognitive Search ã€ã³ããã¯ã¹ãæå®ãããã¿ãŒã³ã«ã€ããŠã¯ãAzure Cognitive Search ãã On Your Data ã«å¯Ÿãã Private Endpoint ãäœæã§ããæ©èœãç»é²å¶ã§å
¬éãããŸããã
ãã ããBlob ã¹ãã¬ãŒãžãæå® ãš ãã¡ã€ã«ã¢ããããŒã ã®ãã¿ãŒã³ã«ã¯ãŸã 察å¿ããŠããŸããã
Storage accounts in private networks are currently not supported by Azure OpenAI on your data.
åè
- Virtual network support & private network support
- Apply Azure Cognitive Search Private Endpoint Request for Azure OpenAI on your data (ç³è«ãã©ãŒã )
ãããã«
ã¢ãã©ã€ã¶ãŒã®èšèªæå®ãªã©ãµãŒãã¹ã®æçã«äŒŽãä»åŸã®æ©èœè¿œå ã«æåŸ
ããéšåããããŸãããçµå±ã®ãšãããã£ãã䜿ãç©ã«ãªãä»çµã¿ãäœãããã«ã¯ãè¶³å
ã®å°éãªããŒã¿æŽåãæãè¿éãšãªãããã§ãã
ãŸãããã£ããæ€çŽ¢ã§æãéãã®æ€çŽ¢çµæãåŸãããªãå Žåã¯ããããããã£ããæ€çŽ¢ã§ããå¿
èŠãããããšåãçŽããŠãAzure Cognitive Search ã®ã¿ã䜿ã£ãåŸæ¥ã®æ
å ±æ€çŽ¢ããã¹ã¢ãŒã«ã¹ã¿ãŒããããªã©æè»ã«å¯Ÿå¿ãããå¿
èŠããããšæããŸãã
以äžã§ããðµ
ã¡ã³ããŒãéžãã Microsoft補åã®ææ°æè¡æ å ±ã»ç¥èŠããå±ãããŸãã â»ãã®Publicationã¯æ¥æ¬ãã€ã¯ããœãããŸãã¯ç±³Microsoftæå±ç€Ÿå¡ã«ããå人ã®èŠè§£ã§ãããæå±ããçµç¹ã®å ¬åŒèŠè§£ã§ã¯ãããŸããã â»Publicationã«åå åžæã®ç€Ÿå¡ã¯ @07JP27ãŸã§ãé£çµ¡ãã ããã
Discussion