ã¯ããã«
ããã«ã¡ã¯ã(æ ª) æ¥ç«è£œäœæã® Lumada Data Science Lab. ã®è«žæ© æ¿å¹žã§ãã
ãã©ã€ããŒããªæŽ»åã«ãªããŸããããKaggleã§ç£šã æ©æ¢°åŠç¿ã®å®è·µåããšããã¿ã€ãã«ã®æ¬ãå·çãã6æã«ç¡äºçºå£²ããŸãããæžç±ã®å·çãšãã貎éãªäœéšãããããšãããããã£ãããªã®ã§ãå·çãéããŠç¥ã£ãããšïŒçµéšããããšããªã©ãæžããããšæããŸããå°æ¥çã«æ¬ãæžããããªãšèããŠããæ¹ã®åèã«ãªãã°å¹žãã§ãã
æ¬èšäºã§ã¯ä»¥äžã®ããšãæžããŠããŸãã
- ãªãå·çããããšã«ãªã£ãïŒããã®ã
- äŒç»ããçºå£²ãŸã§ã®ããã»ã¹
- å·çããéã«ã©ããªããšã§å°ã£ãã
ãŸãã¯æ¬ã®ç޹ä»ã§ãïŒ
èå³ãæã£ãæ¹ã¯æ¯éèªãã§ã¿ãŠãã ããïŒæžåºã«ã䞊ãã§ãŸãïŒïŒ
- [ã¿ã€ãã«] Kaggleã§ç£šã æ©æ¢°åŠç¿ã®å®è·µå -- å®åxã³ã³ããéããããã®æé
- https://www.amazon.co.jp/dp/4865943269
1. ãªãå·çããããšã«ãªã£ãïŒããã®ã
2021幎ã«éšçœ²ã®äººãã¡ãšä»äºãšããŠãå®è·µ ããŒã¿åæã®æç§æžããšããæ¬ãå·çãããã®äžã®ã³ã©ã ã§Kaggleã®ããšãæžããŸããïŒãKaggleã¯æ¥åã«åœ¹ç«ã€ããã¿ãããªè©±ïŒããããèŠãåºçç€Ÿã®æ¹ãããKaggleã®æ¬ãæžããªããããšå£°ãæããããã®ããã£ããã§ãã
ãã ãããã§ã«ããã€ãKaggleæ¬ãããäžïŒããŸããäœã§æ¬æžããã®ïŒããšæã£ãæ¹ãå€ãã®ã§ã¯ãªãããšæããŸãããããæåãªKaggleæ¬ããããããã
ç§èªèº«ããã®è©±ãããã ãããšãã«ãæ£çŽããæããŸããïŒç¬ïŒ
ããã§ãæžãããšã«ããã®ã¯ãKaggleãšãããšãåè² ããã¡ãã«ããé äœããæ°ã«ãªãæ¹ãå€ããšæããã©ããããã ããããªããããåŠç¿ããŒã«ãšããŠåªç§ãªãã ãããšããããšãäŒãããã£ãããã§ãã
ãžã¥ã³ã¯å ã®ãããã«ãæžãããã©ãKaggleã¯æåŒ·ã®åŠç¿ããŒã«ããªãã§ãã
ç§èªèº«ãã³ã³ããå§ãããã£ããã¯ãç§°å·ãã¡ãã«ãåããããã§ã¯ãªãããæ©æ¢°åŠç¿ã®ã¹ãã«ã¢ããããç®çã ã£ãã®ã§ãããããæ©æ¢°åŠç¿ãåŠã³ãããšæã£ãŠããæ¹ããããªãã¬ãã«ã¢ãããç®æãæ¹ã«Kaggleã䜿ã£ãåŠç¿æ³ãåºãç¥ã£ãŠã»ãããšæã£ãŠããŸããã
ãªã®ã§ããããããããã®æé ãèãæ¹ããŸãšããããšã§ããããã人ãã¡ã®èäžãæŒããããèºããå°ãã§ãæžãããããšæããå·çãæ±ºæããŸããã
2. äŒç»ããçºå£²ãŸã§ã®ããã»ã¹
次ã«ãã©ããªããã»ã¹ãçµãŠçºå£²ã«è³ã£ãã®ããç°¡åã«èª¬æããŸãã倧ãŸãã«ã¯äžå³ã®ãšããã§ãã
ãã§ãŒãºâ ïŒäŒç»
ãŸãã¯ãã³ã³ã»ããããèªè
ã¿ãŒã²ããããé¡äŒŒæžç±ãšã®éãããåæã¹ãã«ããšãã£ããšãããå
·äœåããŠãã£ãŠãæçµçã«ã¯ç« ã»ç¯ã¬ãã«ã®èŠåºããšå
容ãããŒãžæ°ãŸã§ã決ããŠãããŸããããããããŸãããããšãç®æ¬¡ãã«ãããéšåã§ãã
ãã®äŒç»æžãåºç瀟ã®äŒç»äŒè°ã«ãããééãããšæŽããŠå·çã«GOãµã€ã³ãåºãŸãã
ãããæåãã€æå€§ã®å£ã ãšæããŸãã
æ¬ãæžããããšæã£ãŠããæ¹ã¯åºç瀟ã«äŒç»æžãæã¡èŸŒãã§ã¿ãŠã¯åŠäœã§ãããã
ãã§ãŒãºâ¡ïŒå·ç
äŒç»æ®µéã®æ§æ³ã«æ²¿ã£ãŠããšã«ããå°éã«æžããŠãããã§ãŒãºã§ãã
ãšã¯èšããäŒç»æ®µéã®ã¡ã¢ã¬ãã«ã®ãã®ããæžç±ã¬ãã«ã«ããŠããã®ã¯èœãšã蟌ãã§ããã®ã¯ããªãèŠåŽããŸããã
ç¹ã«ãPythonã¹ã¯ãªããã亀ããŠèª¬æããŠãã4ç« ä»¥éã§ã¯ãã説æãããããšãå
·äœå â åæã®æµããäœæ â ãããèžãŸãã説æã®ã·ããªãªãäœæ â Pythonã¹ã¯ãªãããçšæ â 説ææãæžããŠããããšããäœæ¥ãããŠããã®ã§ããããããçµæ§å€§å€ã§ããã
ååŸã®ççŸãªããPythonã¹ã¯ãªããã䜿ã£ãŠèª¬æã§ããèšããããã€ã³ãããºã©ããªãããã«ããšæ°ã«ããªããšãããªãããšããŠããçãã§ãã
ã¯ããããããããªæžããŠããã®ã¯ããªãé£æåºŠãé«ãããããŸãã¯ãæžãçŽãåæã§ã²ãšãšããæžãåããããšãåçš¿æã®ç®æšãšããŸãããããšãã°ãåç« ãšççŸãçããŠãåç« ã«æ»ã£ãŠæžãçŽããããã®ããšãã¡ã¢ãæ®ããŠããã«ãšã©ãããšã«ããåã«é²ããããããã£ãŠå
šäœãæžããŠãããšå
šäœåãšåé¡ãã¯ãããªããŠããŸããç§ã«ãšã£ãŠã¯ãã®æ¹æ³ãåã£ãŠãŸããã
ãã ãæžãçŽãåæã§æžãé²ããããšããããä¿®æ£äœæ¥ãåçš¿æãšåãããããã¯ãã以äžããã£ãŠããŸããŸãããäžèŠç¡é§ã«ãèŠããŸãããçµæçã«ã¯å¹ççã ã£ãããªãšã¯æããŸãã
ãã§ãŒãºâ¢ïŒã¿ã€ãã«ã»è¡šçŽ
ã¿ã€ãã«ã衚çŽã¯æ¬æãšéã£ãŠåºç瀟åŽã®ã¿ã¹ã¯ã§ããããã ãæçµçã«ã¯èè
ã®æèŠãåãå
¥ããŠæ±ºå®ããã¹ã¿ã€ã«ã§ããã
èªè
ã«ãšã£ãŠãäžçªæåã«ç®ã«å
¥ãã®ã¯ãã¿ã€ãã«ãšè¡šçŽãã§ãããã¡ãŒã¹ãã€ã³ãã¬ãã·ã§ã³ã§èšããšæ¬æãããéèŠã§ã¯ãªãããšæããŸããäœåºŠãåºçç€ŸãšæåããããŠæ±ºããŸããã
ãã§ãŒãºâ£ïŒPR
ã©ããã£ãŠPRããŠãããã«ã€ããŠã¯ãåºç瀟ã®å¶æ¥ã®æ¹ãšäžç·ã«è°è«ããŠæ±ºããŸããã倧åã¯åºç瀟åŽã®ã¿ã¹ã¯ãšãªããŸãããèè
ã®ã¿ã¹ã¯ãäžéšãããŸãã
ä»åã¯ãç®æ¬å
ã®éžå®ã»äŸé Œãè¬æŒã€ãã³ãã§ã®åç¥æŽ»åãææžããããäœæããªã©ãè¡ãªããŸãããæžåºã§ã®ã€ãã³ããããããšãããããã§ãããã³ããçŠãšããããšãããä»åã¯äœããããŸããã§ããã
3. å·çããéã«ã©ããªããšã§å°ã£ãã
ãããã®ããã»ã¹ãé²ããäžã§ããããããªæ©ã¿ãèŠåŽããããŸãããããããããã®ã§ãããã€ãã«ããŒã£ãŠèª¬æããŸãã
(1) åŠäœã«èªè ã¿ãŒã²ãããåºããã
ãèªè
ã¿ãŒã²ãããã¯ã賌買æ°ãã«çŽçµããéèŠãªèŠçŽ ãªã®ã§ãåºããã¹ãã§ãããããããšã«ããéå£ãåºããŠå
¥éæžã«ããŠããŸããšé¡äŒŒæžç±ãã©ãããŠãå€ããªããŸããäžæ¹ãå·®å¥åã®ããã«äœãã«ç¹åãããšã察象èªè
ãæžããŸãã
ãã¬ãŒããªãã®é¢ä¿ã«ããã®ã§ãããå¡©æ¢
ãèŠã€ããŠããããšã«ãªããŸããäŒç»æ®µéã§ããªãæ©ããšããã ãšæããŸãã
å人çã«ã¯ãæœåšèªè ãå€ãæ¹ãéèŠãªæ°ãããŸãããã®ãããéå£ãåºããïŒå ¥éæžãã«ããã®ãæã£åãæ©ãã§ããäžã®äžã«ãXXXXå ¥éããšããæ¬ãå€ãã®ã¯ããããããšãªãã ãšããããããŠæããŸããã
(2) ã©ããªããšãèªè ãžäŒãããã
æåã«ãæžããŸãããããKaggleãåŠç¿ããŒã«ãšããŠå©çšããããšããæ¬æžã®ã³ã³ã»ãããšããŠããŸãã
ããã«å ããŠãå人çãªçå¿µã»æ¹éãšããŠããç§ãæ©æ¢°åŠç¿ã®å匷ãã¯ãããããã«ããããªæ¬ããã£ããè¯ãã£ãã®ã«ãªããšæããã®ãæžãããšãå¿æããŸããã
ããšãã°ãããŸãã¯ããŒã¹ã©ã€ã³ãäœã£ãŠãã®äžã§è©Šè¡é¯èª€ãããããã¢ãã«ã¯ãŸãã¯LightGBMã䜿ãããªã©åæãããŠãã人ãããããšåœããåã®ããšã§ããå§ããã°ããã®é ã¯ãããåãããŸãããç¹åŸŽéãšã³ãžãã¢ãªã³ã°ã¯ç¢ºãã«éèŠã ãã©ããŸãã¯ãããŒã¹ã©ã€ã³äœæããããªããŒã·ã§ã³èšèšããå
ã ãããšããããšã匷調ããŸãããäºäŸã®ãšãããããã®èª¬æãåãããŠããŸãã
ãã®ãããKaggleã§äžäœã«å
¥ãããã®ãã¯ããã¯ãç¥ãããããšæã£ãŠããæ¹ãããããšæåŸ
ã«ã¯æ²¿ããªãããšæããŸããããããããšãå
¥ããããšæã£ãŠãŸããããããŒãžæ°ããã®ãããããšã«ãªãããã ã£ãã®ã§æ¬æžã§ã¯åºç€çãªãšããã ãã«çµã£ãŠããŸãã
ãã第2匟ãããã°ãããããã®ãæžããããªãšæã£ãŠãŸãã
(3) ã©ããã£ãŠåæã®å®è·µäŸã説æããŠããã
4-6ç« ã§ã¯ãtitanicã³ã³ãããã7ç« ã§ã¯ãHome Creditã³ã³ããã8ç« ã§ã¯ãMLBã³ã³ããã顿ã«ããŠå®è·µäŸãæžããŠããŸã
ãããã100åã®åé¡ãããã°100åã®è§£æ³ãããã100人ã®ããŒã¿ãµã€ãšã³ãã£ã¹ããããã°100åã®è§£æ³ããããŸãããŸããéãããçŽé¢ã®äžã§ã©ã®éšåãã©ããã£ãŠèª¬æãããã¯ããªãæ©ã¿ãŸããã
äžäœè§£æ³ãããŒã¹ã«èª¬æããŠãã£ãŠãè¯ãã®ã§ããããããããåºæ¬ãšãªããããŒã¹ã©ã€ã³ã®äœæãã®ãšãããã¡ã€ã³ã«æžãããšã«ããŸããããã®éšåãªããä»ã®åé¡ã«åãçµãéã«ãå¿çšãå¹ãããæ©æ¢°åŠç¿ã®èãæ¹ã®åºæ¬ãç¥ãããšãéèŠã ãšæã£ãããã§ãã
ã¡ãªã¿ã«ããã®å®è·µäŸã«èŒããã³ã³ãã¯ä»ã«ããæç³»åäºæž¬ã§ãããM5ã³ã³ãããšãäœçœ®æšå®ã³ã³ãã§ãããIndoorã³ã³ãããåè£ãšããŠæããŠããŸããã
ããã«èšããšããããã®Pythonã¹ã¯ãªãããšèª¬ææãäœã£ãŠããŸãããããããè«žäºæ
ããæ³£ãæ³£ãäžžããšããã«ããŸããã
仿ããšãããããå«ãããã£ãšå€ãã®ã³ã³ãã顿ã«ããŠãåé¡ã®æãæ¹ããïŒæ©ãã§ããç¹ãå«ããŠïŒã¢ãããŒãæ¹æ³ãæžããŠãè¯ãã£ããªãšæã£ããããŸããã
(4) å·çã®ããŒã¹é å
鱿«ã®åæ¥ãã¹ãŠãå·çã«è²»ãããšããçæŽ»ãç¶ããŠããã«ãé¢ããããå·çã ãã§äžž1幎ããããŸããã
ããã«æåŸã®3ã¶æã¯ãå¹³æ¥ãä»äºãçµãã£ãŠããå«è¶åºã«è¡ã£ãŠéåºãŸã§æžããšããããšãç¶ããŠãã£ãšçµãã£ããšããæãã§ãã
å«è¶åºã®ã³ãŒããŒä»£ã ãã§ãããããããã£ãŠãŸãïŒç¬ïŒ
æåãåºçç€Ÿã®æ¹ã«èšãããã®ã¯ããç¡çã¯ããªãããŒã¹ã§ããšããããšã§ããã300ããŒãžãæžããšãªããšãçææ±ºæŠã§ã¯ã ãªã§ããã©ãœã³ã¿ãããªé·æçãªå¿æ§ããå¿
èŠã«ãªããŸãã1æ¥åŸ¹å€ããŠã©ãã«ããªããã®ã§ã¯ç¡ãã§ãããã
åœããåã§ãããæ®æ®µã¯ä»äºãããŠããã®ã§ããããã«ã¯ãŒã¯ãããŠãããããªæèŠã§ãããéäžã§æéãåããªããªã£ãŠæ«æãã人ãããŠãå
šããããããªãããªãšæããªãããã£ãŠãŸããã
ããããæžãããšæã£ãŠããæ¹ã¯ãåºç瀟ãšçžè«ããŠç¡çã®ãªãå·çèšç»ãç«ãŠãŠé²ããã°ãšæããŸãã
(5) ã©ã€ãã©ãªã®ããŒãžã§ã³åé¡
ãæžç±ã®ã¹ã¯ãªãããæžããŠå®è¡ãããã©åããªãã£ãããåºåçµæãæžç±ãšéãããšããããšã¯ãç§èªèº«ãä»ã®æžç±ã§äœåºŠãçµéšããŠããŸãã
Python絡ã¿ã®æ¬ã ãšãå°å³ã«ããããããžãã§ãã
å·çäžã®1幎éã«ãè²ã ãªã©ã€ãã©ãªãããŒãžã§ã³ã¢ããããŠããŸããããã®ãããå°å·çŽåã«å šã¹ã¯ãªããã®ãã§ãã¯ãããŸããããã®éã«ãããã€ã倿ããŸããããå šãŠåé¡ãã¯ãªã¢ããŠæºãæããŠçºå£²ãšãªããŸããã
ãããïŒ
æçµçš¿ããçºå£²ãŸã§ã®åæãããã®éã«ãKaggleåæç°å¢ïŒNotebookïŒã§äžéšã©ã€ãã©ãªã®ããŒãžã§ã³ãæŽæ°ãããŠããŸããŸããããã
å
·äœçã«ã¯ã4ç« ã§ãpandas_profilingããšããã©ã€ãã©ãªã䜿ã£ãŠããã®ã§ããããªããšãã®importæã«ãšã©ãŒãåºãŸãããã
ãã®åé¡ã¯ãããŒãžã§ã³ã3.2.0ã«äžããããšã§è§£æ±ºããŸãã
# pandas_profilingãã€ã³ããŒãããåã«å®è¡ã
!pip install pandas_profiling==3.2.0
# ã¢ããããŒãããŠãããšã€ã³ããŒããæåããŸãïŒ
import pandas_profiling as pdp
èªè
ã®ææã§ããã«å€æããã®ã§ïŒããããšãããããŸãïŒãå³åº§ã«åºç瀟ã®HPäžã§å¯Ÿçæ¹æ³ãèŒããŸããã
ãã®åé¡ã¯ãçŸæç¹ã§ã®Kaggleç°å¢ã§çºçããŠããåé¡ãªã®ã§ãäœãšãæ©ãè§£æ¶ããŠã»ããã§ãã
ããŒãžã§ã³åé¡ã®åé¿çãšããŠã¯ãåºæ¥ãã ãã¡ãžã£ãŒã§å®å®ããã©ã€ãã©ãªã䜿ããããããç¡ãããªãšæããŸããã¡ãžã£ãŒãªã©ã€ãã©ãªã§ãããŒãžã§ã³ã¢ããã«ãã£ãŠã¹ã¯ãªããã®äºææ§ãç¡ããªãããšããã®ã§ãéèŠçŽ ããããŸããã
äœããã£ããšãã«ä¿®æ£ãåºæ¥ãªãã®ãæžç±ã®ã³ã¯ã€ãšããã§ããã
(6) æè¡çãªå 容ã®ãã§ãã¯
è«æã®æ»èªã®ãããªãã®ãååšããŠããã®ããªãšæã£ããã©ããããªãã®ãããŸããã§ããã
åºç瀟ã«ãã§ãã¯ã§ãããããªäººãããããã®äººãæžãã°ããããã§ãããèãããšãããªã®ç¡ãã§ãããã
ãæ¬ãæžãããšããããšã¯ãããæå³ããã®äººãäžçªè©³ããããããªã®ã§ã
ãã ããã¯ãæ»èªã®ãããªããšãããªããšäžå®ãªã®ã§ãè·å Žã®ä»²éããäžç·ã«ã³ã³ãã«åºãŠããæ¹ã«ãã§ãã¯ããé¡ãããŸããã
ä»åãååããŠããã ããæ¹ã
ã«ã¯ãæ¬åœã«é ãäžãããŸããã
æ¬ãæžãæ©äŒããã£ããç§ã«å£°ããããŠãã ãããæ©è¿ãããŸããïŒïŒ
ãããã«
æ¬1åã1äººã§æžãã®ã¯ããªããããžãã§ããããããããã1幎ã§ããã
åæã«ã€ããŠã®èãæ¹ãæŽçããæ©äŒã«ããªã£ãããèªåèªèº«ã«ãšã£ãŠãéåžžã«ããã«ãªãçµéšã§ããã
æ¬ãæžããããªãšæã£ãŠããæ¹ã¯ãåºç瀟ã«äŒç»ãæã¡èŸŒãã§ã¿ãŠã¯åŠäœã§ããããã
ç§èªèº«ã¯è¯ãæ¬ãã§ãããšæã£ãŠããã®ã§ããæ©æ¢°åŠç¿ãã¯ãããããªããåæã³ã³ãããããããã ãªããšæã£ãæ¹ã¯ãæ¯éãã®æ¬ãšãšãã«ã¹ã¿ãŒãããŠã¿ãŠãã ããïŒ
æåŸãŸã§èªãã§ããã ãããããšãããããŸããïŒ