R, Python 분석과 프로그래밍의 친구 (by R Friend)

# Importing common libraries

import numpy as np

import pandas as pd

# sample DataFrame

df = pd.DataFrame({'group': ['a', 'a', 'a', 'b', 'b', 'b'], 

                  'value_1': np.arange(6), 

                 'value_2': np.random.randn(6)})

df

# Making GroupBy object

grouped = df.groupby('group')

grouped

<pandas.core.groupby.DataFrameGroupBy object at 0x11136f550>

  (1) GroupBy 메소드를 이용한 집계 (GroupBy aggregation using methods)

count(): 그룹 내 non-NA 개수 

sum(): 그룹 내 non-NA 합 

grouped.count()

grouped.sum() # DataFrame

grouped.sum()['value_2'] # Series

group

a   -0.012473
b    0.072545 

Name: value_2, dtype: float64

pd.DataFrame(grouped.sum()['value_2']) # DataFrame 

min(): 그룹 내 non-NA 값 중 최소값 

max(): 그룹 내 non-NA 값 중 최대값 

grouped.min()

grouped.max()

mean(): 그룹 내 non-NA 값들의 평균값 

median(): 그룹 내 non-NA 값들의 중앙값 

grouped.mean()

grouped.median()

std(): 그룹 내 표준편차

var(): 그룹 내 분산

quantile(): 그룹 내 분위수 

grouped.std() 

grouped.var()

 # interpolation

grouped.quantile(0.1) 

first(): 그룹 내 non-NA 값 중 첫번째 값 

last(): 그룹 내 non-NA 값 중 마지막 값 

grouped.first()

grouped.last()

describe(): 그룹 별 기술통계량 

- 옆으로 길게

 describe().T: 그룹 별 기술통계량 

- 세로로 길게

grouped.describe()['value_1']

grouped.describe()['value_1'].T

  (2) 함수를 이용한 GroupBy 집계: grouped.agg(function)

def iqr_func(x):

    q3, q1 = np.percentile(x, [75, 25])

    iqr = q3 - q1

    return iqr

grouped.aggregate(function) 

grouped.agg(function) 

grouped.aggregate(iqr_func)

grouped.agg(iqr_func)

grouped.quantile([0.75, 0.25])

 산술통계

 통계량 (statistics)

R function 

 중심화 경향

(central

tendency)

 산술평균 (arithmetic mean)

 mean()

 which.max(table())

 기하평균 (geometric mean)

 prod(x)^(1/n)1/mean(1/x)

where, n = length(x)

 연평균성장률 (CAGR

 : Componded Average Growth Rate)

 (FV/IV)^(1/n)-1

where, IV : initial value of an investment
          FV : final value  of an investment
          n : investment periods

 조화평균 (harmonic mean)

 1/mean(1/x)

 가중평균 (weighted average)

 weighted.mean()

 퍼짐 정도

(dispersion)

 분산 (variance)

 var()

 변이계수 (coefficient of variation)

 100*sd(x)/mean(x)

 diff(range())

 IQR (Inter Quartile Range)

 최소값 (min)

 최대값 (max)

 quantile(x, probs=c(,,,,))

 분포형태와

대칭정도

(distribution)

 왜도 (skewness)

 skewness(), fBasics package

 첨도 (kurtosis)

 kurtosis(), fBasics package

 qqnorm(), qqline(), qqplot()

> library(MASS)

> str(Cars93)
'data.frame':	93 obs. of  27 variables:
 $ Manufacturer      : Factor w/ 32 levels "Acura","Audi",..: 1 1 2 2 3 4 4 4 4 5 ...
 $ Model             : Factor w/ 93 levels "100","190E","240",..: 49 56 9 1 6 24 54 74 73 35 ...
 $ Type              : Factor w/ 6 levels "Compact","Large",..: 4 3 1 3 3 3 2 2 3 2 ...
 $ Min.Price         : num  12.9 29.2 25.9 30.8 23.7 14.2 19.9 22.6 26.3 33 ...
 $ Price             : num  15.9 33.9 29.1 37.7 30 15.7 20.8 23.7 26.3 34.7 ...
 $ Max.Price         : num  18.8 38.7 32.3 44.6 36.2 17.3 21.7 24.9 26.3 36.3 ...
 $ MPG.city          : int  25 18 20 19 22 22 19 16 19 16 ...
 $ MPG.highway       : int  31 25 26 26 30 31 28 25 27 25 ...
 $ AirBags           : Factor w/ 3 levels "Driver & Passenger",..: 3 1 2 1 2 2 2 2 2 2 ...
 $ DriveTrain        : Factor w/ 3 levels "4WD","Front",..: 2 2 2 2 3 2 2 3 2 2 ...
 $ Cylinders         : Factor w/ 6 levels "3","4","5","6",..: 2 4 4 4 2 2 4 4 4 5 ...
 $ EngineSize        : num  1.8 3.2 2.8 2.8 3.5 2.2 3.8 5.7 3.8 4.9 ...
 $ Horsepower        : int  140 200 172 172 208 110 170 180 170 200 ...
 $ RPM               : int  6300 5500 5500 5500 5700 5200 4800 4000 4800 4100 ...
 $ Rev.per.mile      : int  2890 2335 2280 2535 2545 2565 1570 1320 1690 1510 ...
 $ Man.trans.avail   : Factor w/ 2 levels "No","Yes": 2 2 2 2 2 1 1 1 1 1 ...
 $ Fuel.tank.capacity: num  13.2 18 16.9 21.1 21.1 16.4 18 23 18.8 18 ...
 $ Passengers        : int  5 5 5 6 4 6 6 6 5 6 ...
 $ Length            : int  177 195 180 193 186 189 200 216 198 206 ...
 $ Wheelbase         : int  102 115 102 106 109 105 111 116 108 114 ...
 $ Width             : int  68 71 67 70 69 69 74 78 73 73 ...
 $ Turn.circle       : int  37 38 37 37 39 41 42 45 41 43 ...
 $ Rear.seat.room    : num  26.5 30 28 31 27 28 30.5 30.5 26.5 35 ...
 $ Luggage.room      : int  11 15 14 17 13 16 17 21 14 18 ...
 $ Weight            : int  2705 3560 3375 3405 3640 2880 3470 4105 3495 3620 ...
 $ Origin            : Factor w/ 2 levels "USA","non-USA": 2 2 2 2 2 1 1 1 1 1 ...
 $ Make              : Factor w/ 93 levels "Acura Integra",..: 1 2 4 3 5 6 7 9 8 10 . 

> # histogram, kernel density curve
> hist(Cars93$MPG.highway, freq=FALSE, breaks=30)
> 
> hist(Cars93$Price, freq=FALSE, breaks=20)
> lines(density(Cars93$Price), col="red", lty=3)

> # mean : mean()

> mean(Cars93$Price)
[1] 19.50968

> # median : median()
> median(Cars93$Price)
[1] 17.7

> # mode : which.max(table())
> Cars93 <- within(Cars93, {
+   Price_cd = character()
+   Price_cd[Price < 10] = "5_10"
+   Price_cd[Price >= 10 & Price < 15] = "10_15"
+   Price_cd[Price >= 15 & Price < 20] = "15_20"
+   Price_cd[Price >= 20 & Price < 25] = "20_25"
+   Price_cd[Price >= 25 & Price < 30] = "25_30"
+   Price_cd[Price >= 30 & Price < 35] = "30_35"
+   Price_cd[Price >= 35 & Price < 40] = "35_40"
+   Price_cd[Price >= 40 & Price < 45] = "40_45"
+   Price_cd[Price >= 45 & Price < 50] = "45_50"
+   Price_cd[Price >= 50 & Price < 55] = "50_55"
+   Price_cd[Price >= 55 & Price < 60] = "55_60"
+   Price_cd[Price >= 60 ] = "60_65"
+   Price_cd = factor(Price_cd, level=c("5_10", "10_15", "15_20", "20_25", "25_30", "30_35", 
+                                       "35_40", "40_45", "45_50", "50_55", "55)60", "60_65"))
+ })
> 
> table(Cars93$Price_cd)

 5_10 10_15 15_20 20_25 25_30 30_35 35_40 40_45 45_50 50_55 55)60 60_65 
   10    23    28    11     8     6     4     1     1     0     0     1 
> 
> which.max(table(Cars93$Price_cd))
15_20 
    3 

문제) 작년 이율이 1%, 올해 이율이 5%인 복리정기예금의 2년간 평균 이율은?

• 산술평균 = (1.01 + 1.05)/2 = 1.03, 즉 3.0%  (삐이~ 잘못된 계산임)
• 기하평균 = (1.01*1.05)^(1/2) = 1.029806, 즉 2.98% (제대로된 계산임)

> # geometric mean
> x <- c(1.01, 1.05) # interest rate of 1st and 2nd year
> prod(x) # 1st year rate x 2nd year rate
[1] 1.0605
> n <- length(x) # length of x vector
> 
> prod(x)^(1/n) # geometric mean
[1] 1.029806

> # CAGR (Compound Average Growth Rate)
> IV <- c(100) # initial value of revenue
> FV <- c(350) # final value of reveune
> n <- c(5) # number of year
> CAGR_rev <- (FV/IV)^(1/n) - 1 # CAGR
> CAGR_rev
[1] 0.2847352

> # harmonic mean : 1/mean(1/x)
> km_per_hour <- c(30, 90)
> 
> arithmetic_mean <- mean(km_per_hour)
> arithmetic_mean # wrong answer
[1] 60
> 
> harmonic_mean <- 1/mean(1/km_per_hour)
> harmonic_mean # correct answer
[1] 45

> # Q1

> # weighted mean
> weighted_earning_rate_1 <- (0.7*0.15 + 0.2*0.09 + 0.1*0.05)/(0.7 + 0.2 + 0.1)
> weighted_earning_rate_1
[1] 0.128
> 
> investment <- data.frame(weight=c(0.7, 0.2, 0.1), earning_rate=c(0.15, 0.09, 0.05))
> weighted_earning_rate_2 <- weighted.mean(investment$earning_rate, investment$weight)
> weighted_earning_rate_2
[1] 0.128

 
> # Q2
> weighted_alcohol_mean_1 <- (200*0.09 + 1000*0.21)/(200+1000)
> weighted_alcohol_mean_1
[1] 0.19
> 
> alcohol <- data.frame(volume=c(200, 1000), alcohol_rate=c(0.09, 0.21))
> weighted_alcohol_mean_2 <- weighted.mean(alcohol$alcohol_rate, alcohol$volume)
> weighted_alcohol_mean_2
[1] 0.19
 

R 기술통계 함수

> x <- c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
> mean(x)
[1] 5.5 

> x <- c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
> median(x)
[1] 5.5
> y <- c(1, 2, 3, 4, 5, 6, 7, 8, 9)
> median(y)
[1] 5

> min(x)
[1] 1
> min(y)
[1] 1 

> my_vec <- c(-5, 3, 10, 3, -12, NA)
> my_vec
[1]  -5   3  10   3 -12  NA
> 
> min(my_vec)
[1] NA
> 
> which.min(my_vec) # index of min value in 'my_vec' vector
[1] 5
> 
> my_vec[which.min(my_vec)]
[1] -12

> max(x)
[1] 10
> max(y) 

> my_vec <- c(-5, 3, 10, 3, -12, NA)
> my_vec
[1]  -5   3  10   3 -12  NA
> 
> max(my_vec)
[1] NA
> 
> which.max(my_vec) # index of max value in 'my_vec' vector
[1] 3
> 
> my_vec[which.max(my_vec)]
[1] 10 

> range(x)
[1]  1 10
> range(y)
[1] 1 9 

> IQR(x)
[1] 4.5
> IQR(y)
[1] 4 

> summary(x)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   1.00    3.25    5.50    5.50    7.75   10.00 

> var(x)
[1] 9.166667
> var(y)
[1] 7.5 

> sd(x); sd(y)
[1] 3.02765
[1] 2.738613 

> install.packages("fBasics") # 왜도, 첨도 분석 가능한 package 설치
> library(fBasics) # package 호출
> hist(mtcars$mpg)
 
 
 
 

 
 
> skewness(mtcars$mpg)
[1] 0.610655
attr(,"method")
[1] "moment"

> kurtosis(mtcars$mpg)
[1] -0.372766
attr(,"method")
[1] "excess"

> sum(x)
[1] 55
> sum(y)
[1] 45

> diff(x, lag=1)
[1] 1 1 1 1 1 1 1 1 1
> diff(x, lag=2)
[1] 2 2 2 2 2 2 2 2
> diff(x, lag=3)
[1] 3 3 3 3 3 3 3 

> # 벡터에 length() 사용 시

> length(x)
[1] 10
> length(y)
[1] 9 

> 
> # 데이터 프레임에 length()사용 시
> length(mtcars)
[1] 11
> 
> # 데이터 프레임의 특정 변수에 length($) 사용 시
> length(mtcars$mpg)
[1] 32

> rank(x)
 [1]  1  2  3  4  5  6  7  8  9 10
> 
> rank(-x)
 [1] 10  9  8  7  6  5  4  3  2  1
>

> mtcars$mpg
 [1] 21.0 21.0 22.8 21.4 18.7 18.1 14.3 24.4 22.8 19.2 17.8 16.4 17.3 15.2 10.4 10.4 14.7 32.4 30.4 33.9 21.5 15.5
[23] 15.2 13.3 19.2 27.3 26.0 30.4 15.8 19.7 15.0 21.4
> 
> rank(mtcars$mpg, 
+      na.last = TRUE, 
+      ties.method = c("max"))
 [1] 20 20 25 22 15 14  4 26 25 17 13 11 12  8  2  2  5 31 30 32 23  9  8  3 17 28 27 30 10 18  6 22
 
> ##-----------------------
> ## rank() {base package}
> ##-----------------------
> 
> # if there are no ties(i.e., equal values), no problem at all
> x <- c(1, 5, 9, 7)
> rank(x) 
[1] 1 2 4 3
> 
> 
> # if there are ties, ties can be handled in several ways
> y <- c(1, 1, 1, 5, 9, 7) 
> 
> # returns average, default setting
> rank(y)
[1] 2 2 2 4 6 5
> rank(y, ties.method = c("average")) 
[1] 2 2 2 4 6 5
> 
> # first occurrence wins
> rank(y, ties.method = c("first")) 
[1] 1 2 3 4 6 5
> 
> # ties broken at random
> rank(y, ties.method = c("random")) 
[1] 3 2 1 4 6 5
> 
> rank(y, ties.method = c("random")) # ...random one more time
[1] 1 3 2 4 6 5
> 
> rank(y, ties.method = c("random")) # ...random...again
[1] 1 2 3 4 6 5
> 
> # rank by max value as used classically
> rank(y, ties.method = c("max")) 
[1] 3 3 3 4 6 5
> 
> # rank by min value as in Sports
> rank(y, ties.method = c("min"))
[1] 1 1 1 4 6 5