Kaggleタイタニック問題の上位4%の解答｜ランダムフォレストを使った解答例

はじめに
Kaggle上位4%の方のアプローチ方針
サブミットの結果

はじめに

Kaggleのタイタニック問題で、正答率80%以上となることが、一つの基準となっています。そこで、Kaggle上位4%の方のNotebookを参考にして、その方法を解説したいと思います。

Kaggle上位4%の方のアプローチ方針

欠損値の多いAgeを補完する（乗客の約20%が欠損）
補完方法は欠損していないデータのNameから欠損データを推定する
推定方法は、ファーストネームが同じ同じ方通しでクルーピングして、その平均値で補完する
これにより欠損数は、約8%に低下するので、それらは平均値で補完
「Age」「Parch」「SibSp」「Fare」をバケッティング

コード

前処理

# imports
from sklearn.ensemble import  ExtraTreesClassifier
import seaborn as sns
import pandas as pd
import numpy as np
import os
train = pd.read_csv("train.csv")
test = pd.read_csv("test.csv")
gender_submission_df = pd.read_csv("gender_submission.csv")

# output
union = [train, test]
passenger_id = []

for i, df in enumerate(union):

    # Age inference based on first name
    name = df['Name'].str.split('.', n=1, expand = True)
    name = name[1].str.split(expand = True)[0]
    name.replace(['(\()','(\))'],'',regex=True, inplace = True)
    df['Name'] = name
    del name
    mean_age = df[['Name','Age']].groupby(['Name']).mean()
    df = df.merge(mean_age, on='Name')
    df['Age'] = df['Age_x'].fillna(df['Age_y'])
    df = df.drop(['Age_x', 'Age_y', 'Name'], axis=1)
    
    # Age fill na as adults
    df['Age'] = df['Age'].fillna(df['Age'].mean())

    # Age bucketing
    age_buckets= [0,2,10,18,60,200]
    age_labels = [0,1,2,3,4]
    df['AgeGroup'] = pd.cut(df['Age'], bins=age_buckets, labels=age_labels, right=False)


    # Parch bucketing
    parch_buckets= [0,1,200]
    parch_labels = [0,1]
    df['Parch'] = pd.cut(df['Parch'], bins=parch_buckets, labels=parch_labels, right=False)

    # SibSp bucketing
    sibsp_buckets= [0,1,200]
    sibsp_labels = [0,1]
    df['SibSp'] = pd.cut(df['SibSp'], bins=sibsp_buckets, labels=sibsp_labels, right=False)
    
    # Fare
    #df['Fare']= df['Fare'].clip(lower= df['Fare'].quantile(0.00), upper= df['Fare'].quantile(0.01), axis=0)
    df['Fare'] = df['Fare'].fillna(df['Fare'].mean())
    
    # Ability to bargain
    #df['Fare'] = df['Fare'].astype(np.int8)
    df['Pclass'] = df['Pclass'].astype(np.int8)    
    df['Ability'] = df['Fare'] / df['Pclass']
        
    # Fare and ability bucketing quartiles
    fare_buckets= [0,23,10000]
    fare_labels = [0,1]
    df['Fare'] = pd.cut(df['Fare'], bins=fare_buckets, labels=fare_labels, right=False)
    
    ab_buckets= [0,4,9,15,20,59,70,10000]
    ab_labels = [0,1,2,3,4,5,6]
    df['Ability'] = pd.cut(df['Ability'], bins=ab_buckets, labels=ab_labels, right=False)
    
    
    # Cleaning
    df = df.sort_values(by=['PassengerId'])
    passenger_id.append(df["PassengerId"])
    df['Sex'] = pd.get_dummies(df['Sex'])
    df['Fare'] = pd.get_dummies(df['Fare'])
    df['SibSp'] = pd.get_dummies(df['SibSp'])
    df['Parch'] = pd.get_dummies(df['Parch'])
    df = df.drop(['Embarked', 'PassengerId', 'Ticket', 'Age', 'Cabin'], axis=1)
        
    union[i] = df

モデル生成

x_train = union[0].drop("Survived", axis=1)
y_train = union[0]["Survived"]
x_test  = union[1]

# Extremely randomized trees
ex = ExtraTreesClassifier()
ex.fit(x_train, y_train)
y_pred = ex.predict(x_test)
ex.score(x_train, y_train)
score = round(ex.score(x_train, y_train) * 100, 2)
print('Extremely Randomized Trees', score)

サブミット

#submission
submission = pd.DataFrame({
        "PassengerId": passenger_id[1],
        "Survived": y_pred
    })
submission.to_csv('my_submission_Extremely randomized trees.csv', index=False)