ã¯ããã«
ããã«ã¡ã¯ã(æ ª) æ¥ç«è£œäœæã® Lumada Data Science Lab. ã®è«žæ© æ¿å¹žã§ãã
æè¿ãAIæè¡ã®é²æ©ã«ãããæ§ã
ãªã¿ã¹ã¯ãå¹çåãããŠããŸãããã®äžã§ãããChatGPTã ã¯èªç¶èšèªåŠçã®åéã§é©æ°çãªææãäžããŠããã倧ããªæ³šç®ãéããŠããŸãã
ä»åã¯ããã®ChatGPTãæŽ»çšããŠarXivè«æã®èŠçŽãçæããç°¡æããŒã«ãäœæããŠã¿ãã®ã§ããã®ããã»ã¹ãå
±æããŸããæ¬èšäºã®æ§æã¯ä»¥äžã®ãšããã
- ChatGPTã«ããããããšãäŒããŠã¿ã
- çæã³ãŒããä¿®æ£ããŠæåŸ éãåãããã«ãã
- ããŒã«åãèããŠããã
ãšã«ãã ãChatGPTã«ç©æ¥µçã«é ŒãïŒã ãã³ã³ã»ããã«ããŠäœããŸããã
忥ã«è¶£å³ã§ãã£ãã®ã§ããŒã«ã®ã¬ãã«ãå®çšæ§ã«ã€ããŠã¯æ·±ãèããªãã§ãã ããïŒ
ãã®åã«èªå·±ç޹ä»ïŒïŒæ¬ã®ã¢ããŒã«ïŒ
ç§ã¯æ¥ç«è£œäœæã§é¡§å®¢èª²é¡ãããŒã¿åæã䜿ã£ãŠè§£æ±ºããæ¥åãè¡ã£ãŠããŸãã
â 玹ä»ããŒãž ïŒyoutubeåç»ïŒ
è¶£å³ã¯åæã³ã³ãã§ããæžç±ãæžããŠããã®ã§è¯ããã°èªãã§ã¿ãŠãã ããïŒ
- [æžå] Kaggleã§ç£šã æ©æ¢°åŠç¿ã®å®è·µå -- å®åxã³ã³ããéããããã®æé
- https://www.amazon.co.jp/dp/4865943269
1. ChatGPTã«ããããããšãäŒããŠã¿ã
ãŸãã¯ãéçºãããããŒã«ã®æŠèŠãChatGPTã«äŒããŸããä»åã¯ãarXivè«æã®èŠçŽãçæããããŒã«ããäœæããããšã«ããŸãããèæ¯ãšããŠã¯ãæè¡æ
å ±ã調æ»ããéãæ°å€ãã®arXivè«æããéèŠãããªãã®ãå¹ççã«èŠæ¥µããããªãšæã£ãããã§ãã
ãšããããšã§ãå®çŸããŠã»ããããšãããã¹ãã§æžããChatGPTã«æããããŠã³ãŒããçæããŠããããŸããã
å ·äœçã«ã¯ããããªããã³ããïŒïŒChatGPTã«æç€ºããããã¹ãã®ããšïŒãæããããŸããã
### ããããããš ###
1. å©çšè
ãããŒã¯ãŒããå
¥å
2. å
¥åããããŒã¯ãŒãã«é¢é£ããè«æã®pdfãarxivããããŠã³ããŒã
3. ããŠã³ããŒãããpdfãã¡ã€ã«ãããã¹ãããŒã¿å
4. ChatGPT APIã䜿ã£ãŠè«æã®å
容ãèŠçŽããŠç®æ¡æžããã
5. è«æã®ã¿ã€ãã«ïŒURLïŒèŠçŽïŒç®æ¡æžãïŒïŒã衚ã§ãŸãšããŠåºåãã
### å©çšããŒã« ###
- python
- gradio
- ChagGPT API
ãããå®çŸããããŒã«ãäœã£ãŠãã ããã
ãããªãµãããããæ¥æ¬èªã®ããã¹ãã§ãã以äžã®ãããªããã£ãœãã³ãŒããçæããŠãããŸãã
import gradio as gr
import openai
import arxiv
import PyPDF2
from io import BytesIO
import requests
# APIããŒã®èšå®
openai.api_key = "<your_api_key>"
def summarize_paper(keyword):
# è«æã®æ€çŽ¢ãšããŠã³ããŒã
search_query = f"all:{keyword}"
results = arxiv.query(search_query, max_results=3)
summaries = []
for result in results:
# PDFãããŠã³ããŒã
pdf_url = result['pdf_url']
response = requests.get(pdf_url)
pdf_file = BytesIO(response.content)
# PDFãããã¹ãã«å€æ
pdf_reader = PyPDF2.PdfFileReader(pdf_file)
text = ""
for page_num in range(pdf_reader.numPages):
text += pdf_reader.getPage(page_num).extractText()
# è«æã®èŠçŽãçæ
prompt = f"Summarize the following paper in bullet points:\n{text}"
response = openai.Completion.create(
engine="text-davinci-002",
prompt=prompt,
max_tokens=150,
n=1,
stop=None,
temperature=0.7,
)
summary = response.choices[0].text.strip()
# ã¿ã€ãã«ãURLãèŠçŽãä¿å
summaries.append({
"Title": result['title'],
"URL": pdf_url,
"Summary": summary
})
return summaries
def display_summary(summaries):
output = ""
for summary in summaries:
output += f"Title: {summary['Title']}\n"
output += f"URL: {summary['URL']}\n"
output += f"Summary:\n{summary['Summary']}\n\n"
return output
iface = gr.Interface(
fn=summarize_paper,
inputs=gr.inputs.Textbox(lines=1, label="Keyword"),
outputs=gr.outputs.Textbox(label="Summarized Papers"),
title="Paper Summarizer",
description="Enter a keyword to search for related papers on arXiv, and get summarized bullet points for each paper.",
)
iface.launch()
ãããŒããšããããããã£ãœãïŒïŒ
ãšæããŸããããæ®å¿µãªãããã®ã³ãŒãã¯ãšã©ãŒã ããã§åããŸããããã
2. çæã³ãŒããä¿®æ£ããŠæåŸ éãåãããã«ãã
次ã«ãChatGPTããåŸãããã³ãŒããããšã«ãæåŸ
éãã®åäœãããããã«ä¿®æ£ãå ããŠãããŸããçæããã³ãŒãã¯ãããã£ãœããã§ããããšã«ããåããŸããããããŸããåºå圢åŒã衚ã«ãªã£ãŠããªããªã©ãäžéšæåŸ
ããåŠçã«ãªã£ãŠãŸããã§ããã
ãã®ãããã¯ç§ã®ããã³ããã®æžãæ¹ãæªããããããããŸããã
ããã§ãæåŸ éãã«åããããã«ãããã€ãä¿®æ£ããŸãããä¿®æ£ç¹ã3ã€ã«çµã£ãŠèª¬æããŸãã
2-1. ã©ã€ãã©ãªã»é¢æ°ã®ä¿®æ£ïŒç¥èãè¥å¹²å€ãïŒ
ã³ãŒãå®è¡æã«ãšã©ãŒãåºãå Žåããšã©ãŒã¡ãã»ãŒãžãChatGPTã«ã³ããããã°çŽãæ¹ãŸã§åçããŠãããŸãã
ããããChatGPTã®åŠç¿ã§ã¯2021/9ãŸã§ã®ããŒã¿ãã䜿ã£ãŠããªããããé©åãªåçãåºæ¥ãªãå ŽåããããŸãã
ããšãã°ãä»åã®äŸã ãšãarxiv.query()ã¯çŸè¡ããŒãžã§ã³ã§ã¯ååšããªããããarxiv.Search()ã«å€ããå¿
èŠããããŸãã
ããã¯ãä»ã®ChatGPTã«ãããèããŠãæ£ããããæ¹ãè¿ã£ãŠããªãã®ã§ãå
¬åŒããŒãžãGoogleæ€çŽ¢ã§èª¿ã¹ãŠå¯Ÿå¿ããŸããã
ã¡ãªã¿ã«ãChatGPTã¯ãååšããªã颿°ããå¹³æ°ãªé¡ããŠåçããŠãããããŸãã
ç¥ããªããŠãæããªãã®ã§ãçŽ çŽã«ãåãããŸããããšèšã£ãŠã»ããã§ãããã
2-2. èŠçŽãæç€ºããããã³ããã®ãã©ãã·ã¥ã¢ãã
è²ã
ãªç®æãä¿®æ£ããããšã§ãã³ãŒãããšã©ãŒãªãå®è¡ã§ããããã«ã¯ãªããŸãããããããç§ã®ã€ã¡ãŒãžãšããŠã¯çãæç« ã§ãã€ã³ãã ããåºåããŠãããã€ã¡ãŒãžã ã£ãã®ã§ãããçæãããèŠçŽæããæ³å®ããŠãããããé·ãããšãã課é¡ããããŸããã
ãããç§ã®æåã®ããã³ãããæªãã£ãã®ã ãšæããŸãã
ããã§ã以äžã®ããã«çްããæç€ºããããã«å€ããŸãããïŒ{text}ã¯è«æã®æ¬æã§ãïŒ
ã©ã®ãããªããã³ãããæžãã°æåŸ
ãããããªèŠçŽæãçæãããããåãããªãã£ãã®ã§ããããã³ããäœæ -> èŠçŽæã®çæ -> æåŸ
ããèŠçŽæãã確èªããšãããµã€ã¯ã«ãäœåºŠãç¹°ãè¿ããŠãæçµçã«ãã®ãããªããã³ããã«å°éããŸããã
ãªããªãèšãããšãèããªãã®ã§ã¡ãã£ãšïŒããªãïŒïŒèŠåŽããŸããã
ãªãããã®ããã³ããã䜿ã£ãŠããæåæ°å¶éãè¶
ããããå¥èªç¹ã䜿ã£ãŠãããããŠãå®å
šã«ã¯èšãããšãèããŠãããªãã§ãã
ãã®ãããã¯ããŠããŠãå¿
èŠããã§ããã
prompt = """
### æç€º ###
è«æã®å
容ãçè§£ããäžã§ïŒéèŠãªãã€ã³ããç®æ¡æžãã§3ç¹æžããŠãã ããã
### ç®æ¡æžãã®å¶çŽ ###
- æå€§3å
- æ¥æ¬èª
- ç®æ¡æžã1åã20æå以å
- ã«ã³ãã¯äœ¿çšçŠæ¢
- ãããã¯äœ¿çšçŠæ¢
- æç« ã®çµããã¯äœèšæ¢ãïŒäœèšæ¢ãã®äŸïŒ ïœïœã蚱容ãã â ïœïœã蚱容ïŒ
### 察象ãšããè«æã®å
容 ###
{text}
### åºååœ¢åŒ ###
ã¿ã€ãã«(åå)
- ç®æ¡æžã1
- ç®æ¡æžã2
- ç®æ¡æžã3
"""
ããšæ³šæç¹ãšããŠã¯ãã¢ãã«ã«ãã£ãŠå
¥åã§ããããŒã¯ã³é·ã®äžéãæ±ºãŸã£ãŠããã®ã§ïŒgpt-3.5-turboã ãš4,096ããŒã¯ã³ïŒããããè¶
ããæã¯å
¥åã§ããŸããããã®ãããè«æã®å
šæã§ã¯ãªããåé ã®ãµããªéšåã®ã¿ãæœåºããŠç®æ¡æžãããããã«å€ããŸããã
ãŸããã¢ãã«ãtext-davinci-002ãšãªã£ãŠããã®ã§ãgpt-3.5-turboã«å€ããŸããã
# ChatGPT APIãåŒã³åºãã³ãŒã
response = openai.ChatCompletion.create(
model="gpt-3.5-turbo",
messages=[
{"role": "system", "content": "ããªãã¯ããŒã¿ãµã€ãšã³ãã£ã¹ãã®çç·Žè
ã§ããååŠè
åãã«åãããããæããã®ãåŸæãªã¹ãã·ã£ãªã¹ãã§ãã"},
{"role": "user", "content": prompt},
],
temperature=0.0,
)
2-3. å ¥åºåç»é¢ã®èª¿æŽ
ChatGPTã«ãåºåã衚ã«ããŠããšãé¡ãããŠããŸãããããèŠããšå
šç¶ãããªã³ãŒãã«ãªã£ãŠãŸãããããã«ã€ããŠã¯å床ããã匷調ããŠChatGPTã«äŸé ŒããŠã³ãŒããæžããŠããããŸããã
æ¹æ³ãšããŠã¯ãèŸæžåã®åºåçµæãHTML圢åŒã«å€æããããã«Gradioã®outputsãHTML圢åŒã«å€æŽããŸãããGradioã䜿ã£ãããšããªãã£ãã§ãããäœ¿ãæ¹ãå«ããŠChatGPTãšçžè«ããªããé²ããŸããã
ã³ãŒããã©ãæžãçŽããããããã質åããã°ãããã«ä¿®æ£ã³ãŒããæç€ºããŠãããã®ã§æ¬åœã«é Œãã«ãªããŸãïŒåããªãã³ãŒããªããšããããŸããïŒã
# ã€ã³ã¿ãŒãã§ã€ã¹ã®ã¬ã€ã¢ãŠããã«ã¹ã¿ãã€ãº
iface = gr.Interface(
fn=summarize_paper,
inputs=[
gr.components.Textbox(label="æè¡æ
å ±ã®ããŒã¯ãŒã"),
gr.components.Textbox(label="æ€çŽ¢è«ææ°"),
gr.components.Radio(choices=["gpt-3.5-turbo", "gpt-4"], value="gpt-3.5-turbo", label="å©çšããçæAIã¢ãã«"),
gr.components.Slider(minimum=0, maximum=1, label="temperature"),
],
outputs=gr.components.HTML(label="Summarized Papers"),
title="arXivè«æèŠçŽAI (by Moro)", #"arXivã®èª¿æ» (by Moro)",
description="調ã¹ããæè¡æ
å ±ã®ããŒã¯ãŒããšïŒããã¯ã¢ããããŠèŠçŽããè«ææ°ãæå®ããŠãã ããã\n - AIã§é¢é£ããŒã¯ãŒããèªåçæããŠæ€çŽ¢ã«å©çš \n - æ€çŽ¢ã§ãããããè«æãèŠçŽããŠãã€ã³ããç®æ¡æžã",
theme="default",
allow_flagging='never',
)
äœæããWebã¢ããªã®ç»é¢ïŒ

3. ããŒã«åãèããŠããã
ãã£ãããªã®ã§ãããŒã«ã®åç§°ãChatGPTã«èããŠããããŸãããæ©èœã®æŠèŠãå ¥ããŠããããã€ãã®ããŒã«åç§°ã®ã¢ã€ãã¢ãåºããŠããšãé¡ãããŠã¿ãŸããã
- GPT-IntellectWave: ArXiv Scholarly Navigator
- ArxivSage: ChatGPT-Enhanced Academic Assistant
- GPT-Cerebrum: ArXiv Wisdom Explorer
- ArxivMentor: AI-Powered Knowledge Curator
ãïŒããã£ã±ãæ¥æ¬èªããããªã
- ArxivããŒã¯ãŒã&èŠçŽãã¹ã¿ãŒ: ChatGPTç ç©¶ã¢ã·ã¹ã¿ã³ã
- Arxivç¥ã®é£é: AIæèŒããŒã¯ãŒãçºèŠãšè«æèŠçŽ
- Arxivè«æããã¯ã¢ãã&èŠç¹æŽç: AIæèŒç ç©¶ãµããŒã¿ãŒ
- ArxivèŠçŽãšãã¹ããŒã: è«æããã¯ã¢ãããšèŠç¹æŽçã®AIã¬ã€ã
ãïŒãé·ããã
- arXivæè¡æ
å ±åéã¢ã·ã¹ãAI
- arXivAIæè¡æ
å ±åéã¢ã·ã¹ã
- arXivæè¡æ
å ±åéã¢ã·ã¹ã+: AIæèŒ
- arXivæè¡æ
å ±åéã¢ã·ã¹ãPro: AIçæ
ã·ã³ãã«ãªæ¹ããã£ã±ãããæ°ãããã
ãšããããšã§ãæçµçã«ã¯ãarXivè«æèŠçŽAIãã«ããŸãããæçµå€æã¯ãã¯ãäœãæã®ææ§ã§ããã
ãããã«
ä»åç°¡åãªããŒã«ã§ãããChatGPTãšååããŠ1ã€ã®ããŒã«ãéçºããŠã¿ãŸããã
Webã¢ããªéçºæªçµéšã§ããããäœãšãåããã®ãäœæã§ããŸããã忥ãããã§å®æããŠããã®ã§ãããªãå¹çåã§ããŠããšæããŸãã
ãšããããChatGPTç¡ãã ã£ããåããã®ãäœããã«æ«æããŠããæ°ãããŸãã
ããããããšãæœè±¡çãªã¬ãã«ã§æžãã ãã§ãã³ãŒãçæãŸã§ããŠãããã®ã¯æ¬åœã«é©åœã§ãã
ãã®äžã现ããçžè«ãã§ããããå匷ã峿¹ãã§ããæ°æã¡ã§ãããã ãæ¬åœã«æåŸ
ããããšãå®çŸã§ãããšãããŸã§æã£ãŠããã«ã¯ã䜿ãæåŽã«ãããçšåºŠã®ç¥èãããŠããŠãå¿
èŠã ãªãšãæããŸããã
ChatGPTãå®éã«äœ¿ã£ãŠã¿ãŠããããªããšãåºæ¥ããã ãšãããããªã®ãäœã£ãŠã¿ãããšããããã«æ³åãšæåŸ
ããµããã¿ãŸããããã®ãããªAIæè¡ãå©çšããããšã§ãã¿ã¹ã¯ã®å¹çåã ãã§ãªããã¢ã€ãã¢åµåºã»æ°ããªçºèŠã驿°ã«ãã€ãªãããšæããŸããã
æåŸãŸã§èªãã§ããã ããããããšãããããŸããããã®èšäºããChatGPTãæŽ»çšããããŒã«éçºã«èå³ãæã€æ¹ã
ã«ãšã£ãŠã貎éãªæ
å ±ãã€ã³ã¹ãã¬ãŒã·ã§ã³ãæäŸã§ããŠãããå¬ããã§ãã
ã¡ãªã¿ã«ãåœç¶ãªããïŒããã®èšäºãåæ¡ã¯ChatGPTã«æžããŠããã£ãŠãŸãããµãã£ãšããæç€ºã§ãããªæãã®æç« ãçæã§ããã£ãŠãã³ãã«ãããã§ããã