
ããã㪠rinna ã LoRA ã§ã·ã¥ããš
ããŠçãããChatGPTã䜿ãã®æ¥œããã§ãããïŒ
ã§ã䜿ã£ãŠããã¡ã«æããŸããïŒ
俺ã ãã®æåŒ·ã® LLM ããããåãæ¬²ãããããïŒïŒ
ã¯ãã
Newby ã§ããç§ããå€åã«æŒãããã®ã¹ããªãŒã ã«èº«ãä»»ããŠããŸããŸããã
ãšãããã以å㯠rinna ã CALM ãã·ã¥ããšåãããŸããã
ãã®æã¯ Colab ãç¡èª²éã ã£ããªãããã(ãïŸÐŽïŸ)ïŒé ãç®ïŒ
ã¯ãã
ããã¯ããŠãããäºååŠç¿æžã¿ã®ããŒã¹ã¢ãã«ãåããããšèªäœã¯çµæ§ç°¡åãªã®ã§ããã
ã§ã¯æ¬¡ã«ããã¹ãã¯ãããïŒ
Yes !! Fine-Tuning !!
ãšããããã§ãã¥ãŒãã³ã°ã詊ãããšãã話ã§ããïŒå眮ããé·ãïŒ
7b ã¯ã€ãã
ããŠãGoogle Colab ã Google Drive ã«èª²éã¯ããŠç°å¢ã¯ããã
ã§ã¯åŸã¯å¯Ÿè±¡ã¢ãã«ã決ããã®ã¿ãããïŒïŒ
ãšããããšã§ãªããšãªãååãšã¢ãŒããã¯ãã£ãç·å㪠RWKV-RAVEN-7b ããã£ãŠã¿ãããšæã£ããããããã
å¯ãã®ããããã®ããªãã¯æ°ã¥ããã§ãããããã
ãã®éžæãééãã§ãããšããããšã«ããã
ãŸããçµè«ããèšããšãéãšæéãããããçµå±çµãããã«ã¡ã¢ãªãå¹ãé£ãã§ã¯ã©ãã·ã¥ãã (Žºâºïœ)
çµéšè ã®æ¹ã«å°ããã¢ãã«ãããã£ãæ¹ãããããšã¢ããã€ã¹ãããã ããŸããããã
ãããããã ãããããªã§ãã£ãŠã¯ãããªããããããçµéšã«ãªããŸããããã
ãšããããšã§çµæ§ãã£ãŠã人ãå€ã rinna 3.6b ããã¥ãŒãã³ã°ããããšã«ããã®ã§ããã
æ®éã«ãã£ãŠãã€ãŸããªãããïŒ
ããŠãrinna ããã¥ãŒãã³ã°ãããšãã£ãŠãæ®éã«ãã£ãã ãããã€ãŸãããªãã
ãªããŠã
ã®ã§ãLoRA ãæ¡çšããŠæ§èœæ¯èŒã¯æãæšãŠãŠé¢çœãå£èª¿ã«ãªããã©ãããæ€èšŒããããšã«ãªã£ãã®ã§ããïŒïŒ
ãšãªãã°ãŸãã¯ã·ã¥ããšå å·¥ã§ããããŒã¿ã»ãããæ¢ããã°ãªãã¬ã
ããã§ãã¡ã ã¯ãããã ããã®ããŒã¿ã»ãããå©çšãããŠããã ããŸããïŒïŒæè¬ãïŒïŒ
ã§ããã¡ãã®ããŒã¿ã»ãããã·ã¥ããšå å·¥ããããã§ãããã©ãããŸãããããããïŒ
ããŒãïŒ
ãšãããã ããã å£èª¿ã«ããŠã¿ãïŒ
ãšããããšã§æ©æ¢°çã«èªå°Ÿã眮æããŠããŒã¿ã»ãããã·ã¥ããšäœã£ãŠã¿ãŸããã
å ããŒã¿ãšåãã CC BY SA 3.0 ã©ã€ã»ã³ã¹ã§ãã€ããšçœ®ãã
åŸã¯ããã䜿ã£ãŠ Let's tuninng ïŒïŒ
ããåã
ãšãã£ãŠã rinna ã®ãã¥ãŒãã³ã°ã«é¢ããŠã¯ npaka 倧å çã®èšäºããŸãã£ãšç䌌ãããŠããã ããã®ã§ç¹ã«æžãããšã¯ããããŸããïŒïŒ
ãã¡ãããåç §ãã ããïŒïŒ
ããã£ãïŒ
ã¯ãããšããããã§ãã¥ãŒãã³ã°å·¥çšã¯è¯éºã«ã¹ã«ãŒã
æã ãæ¬²ããã®ã¯çµæã ãçµæÂ ïŒã-ããœïŒã¯ã€ã
npaka ããã®èšäºã§ã¯ããããåçãããŠããã
generate("èªç¶èšèªåŠçãšã¯ïŒ")人éãæ¥åžžçã«äœ¿ã£ãŠããèªç¶ãªèšèªãçè§£ããèªç¶ã«æç« ãçæããæè¡ãgenerate("æ¥æ¬ã®éŠéœã¯ïŒ")æ±äº¬ã¯ãæ¥æ¬ã®éŠéœã§ããgenerate("ãŸã©ãâãã®ã«ã§äžçªããããã®ã¯ïŒ")æçŸã»ããã¯ããŸã©ãããã®èŠªåã§ãããé女ã®äœ¿ãéã§ãããã¥ãŠããšã®åã§ã
æéãé¡ããèªåã®åœãæã£ãŠããããŠçµæã¯ãããïŒ
ããã£ãïŒïŒ
generate("èªç¶èšèªåŠçãšã¯ïŒ")Natural Language Programmingãšã¯ãèªç¶æã®ããã¹ããåæãã
æå³ã®ããæ
å ±ã«å€æããããã»ã¹ã®ããšã§ãã
èªç¶èªåŠçã¯ãæ©æ¢°åŠç¿ãšãã¿ãŒã³èªèã®åéã«ãããç ç©¶ãšå¿çšã®é²æ©ã§ããã
人éã®èªç¶ãªèšèªäœ¿çšãã¢ãã«åããã³ã³ãã¥ãŒã¿ã§èªç¶ã«å®è¡ã§ããããã«ããã
< NL ãã®ããã»ã¹ã¯2ã€ã®ã¹ãããã«åãããŠããŸã:ãŸããããã¹ãã®æ§æãšæå³ãè§£æã(æ§ææ³)ã
次ã«ãæ§æ³ãšæ§èªã®é¢ä¿ãåŠç¿ãããæå³ããçæãã(æå³è«)ãæ§æ³ã®åŠç¿ã¯äžè¬ã«ã
åèªããã¬ãŒãºã®æèã«ãããæå³çé¢ä¿ãç¹å®ããæèè§£æ(æèæå³è§£æ)ãšã
ç¹å®ã®æ§æ³ãç¹å®ãåé¡ããæå³åé¡(圢æ
çŽ è§£æãšææ³è§£æã®çµ±å)ããæããŸãã§ããããgenerate("æ¥æ¬ã®éŠéœã¯ïŒ")æ±äº¬ã¯ãæ¥æ¬ã®éŠéœã§ãããäžçã§æã人å£ã®å€ãéœåžã§ããããgenerate("ãŸã©ãâãã®ã«ã§äžçªããããã®ã¯ïŒ")ãããã¡ãããã»ããã¡ããã§ããããèšã£ãŠãããšã¯ãã¡ããã¡ãã ãã©ã¡ãããšããã£ãŠã(*ÂŽâœïœ*)
æµç³ã¯ LoRA ïŒïŒéã¿ã¥ãæäœã®ãŠã£ã¶ãŒãã§ããªïŒïŒ
åå rinna ã®æã«äœ¿ã£ãã¯ãšãªãæããŠããã
generate("LLM ãšãããŸã㊠AI ãšãšãããã®å¿ã¯ããã")AIãšAIã§ãã§ãããããããã ã
generate("2023幎ã®äžçªã»ã³ã»ãŒã·ã§ãã«ãªãããã¯ã¯ïŒ")2022幎ã2023幎ã«èµ·ãã£ãæãã»ã³ã»ãŒã·ã§ã³ãªåºæ¥äºã¯ãç±³åœãå®å®è»ãåµèšããããšã§ãã
å®å®è»ã¯ãå®å®ç©ºéã§ç±³åœã®åœå®¶å®å
šä¿éãå®ãã
å°çäžã®ä»ã®åœã
ã«è
åšãäžããç©äœã远跡ããããšãç®çãšããŠããŸãã§ããããæ¥ãª SF ããã
generate("ãã¬ãã¢ã«äžä»£ã«ããããã®ã¢ãã¡ãšããã°ããã")ãœãŒãã¢ãŒãã»ãªã³ã©ã€ã³ãããœãŒãããæ¶Œå®®ãã«ãã®æé¬±ãããšãŽã¡ã³ã²ãªãªã³ãããã©ãŽã³ããŒã«ã
ãã¹ã©ã ãã³ã¯ããã¯ã³ããŒã¹ããNARUTO-ãã«ã-ããéŒã®é¬éè¡åž«ãããã«ãã¢åœç©èªã
ãã¹ã¿ãŒãŠã©ãŒãºããããªãŒããã¿ãŒãããã±ã¢ã³ããã«ãŠããŒã€ããããããæ°äžçŽãšãã³ã²ãªã²ã³ã
ã忢åµã³ãã³ãããã£ããã³ã»ã¢ã¡ãªã«ããã¢ã€ã¢ã³ãã³ããã¢ãã³ãžã£ãŒãºã
ãã¹ãã€ããŒãã³ãã§ãããããšãŽã¡ã³ã²ãªãªã³äºåã§ãŠããããw
generate("ä»ãã¹ã¹ã¡ã® VTuber ãšããã°ããã")VTuverãVtuberãããŒãã£ã«YouTuberã3DCGãMirrativãYouTubeãAnimojiãLive2DãUnityã
Cocos2d-xãFirebaseãGoogle Cloud PlatformãSlackãLinkedInãBitbucketãPythonãNode.js
ãRubyãJavaãC++ãPHPãHTMLãFlashãJSãJAVAãKotlinãAndroidãiOSãEclipseã
TypeScriptãSwiftãHaxeãPromiseãDockerãDjangoãObjective-CãServerlessã
SpringãReactãFluxãReduxã§ããããããããããåèªã¯ããããïŒ
generate("ChatGPT ã«ã€ããŠã©ãæãïŒ")ãã£ããGPTTã¯ãC2CããŒã±ãããã¬ã€ã¹ã®1ã€ã§ã
ãŠãŒã¶ãŒãååã売買ããããã«äœ¿çšãããã©ãããã©ãŒã ã§ãã
ãã®ãã©ãããã©ãŒã ã¯ã2011幎ã«èšç«ãããŸããã§ããããåŠæ³ãéããããã
generate("ããªã㯠rinnna ã«ã€ããŠäœãç¥ã£ãŠãïŒ")Linnaã¯ãRyan D. Ericksonã«ãã£ãŠéçºããããWebããŒã¹ã®ãã£ãããããã§ãã
ãŠãŒã¶ãŒã¯Linaã«è³ªåãããããšãã§ããLineã®äŒè©±ã®ãããªãããšããããŸãã§ããããå¥äººãããïŒïŒ
åçãç Žå€©èããããïŒïŒ
ã§ãèªå°Ÿããããããèš±ãïŒïŒ
ãšããããã§ãç¹ã«ãããšèšã£ãŠçšéã¯ãªããã©ç§ã¯æºè¶³ããã
ããããããã¥ãŒãã³ã°ããŠéãã§ããããæåã
ãšããããã§ãããŸãã
ç¶ç·šãã§ãã