Commit e5ca5596 authored by Yuanle Song's avatar Yuanle Song
Browse files

v1.2.2 fix callHistory memory leak issue 

In old code, when call history api enabled, if no one calls that api,
history record thunks will build up in memory. This problem is fixed
now.
parent b8c26644

Lib.hs

+2 −2
Original line number Diff line number Diff line
@@ -10,6 +10,6 @@ boundedPushRight :: S.Seq a -> Int -> a -> S.Seq a 
boundedPushRight buffer maxSize item =

    if S.length buffer == maxSize

    then

        (S.|>) (S.drop 1 buffer) item

        S.drop 1 buffer S.|> item

    else

        (S.|>) buffer item
 
        buffer S.|> item

Main.hs

+1 −1
Original line number Diff line number Diff line
@@ -68,7 +68,7 @@ writeCallHistoryMaybe rc rawIP = 
    when (rcServeHistoryPage rc) $ do

      time <- liftIO getCurrentTimeCST

      userAgent <- header "User-Agent"

      callHistory <- liftIO $ takeMVar (rcCallHistory rc)

      !callHistory <- liftIO $ takeMVar (rcCallHistory rc)

      let newEntry = CallHistory {

                       chTime = time

                     , chClientIP = rawIP

get-client-ip.cabal

+2 −1
Original line number Diff line number Diff line 
name:          get-client-ip

version:       1.2.1

version:       1.2.2

cabal-version: >= 1.8

build-type:    Simple
@@ -9,6 +9,7 @@ executable get-client-ip 
    other-modules:  Lib

    ghc-options:    -Wall -threaded -O2 -rtsopts -with-rtsopts=-N

    extensions:     OverloadedStrings

                  , BangPatterns

    build-depends:  base   >= 4      && < 5

                  , scotty

                  , text

operational

+47 −1
Original line number Diff line number Diff line 
* COMMENT -*- mode: org -*-

#+Date: 2019-04-02

Time-stamp: <2019-04-03>

Time-stamp: <2019-09-03>

#+STARTUP: content

* notes                                                               :entry:

** 2019-04-02 how to deploy get-client-ip?				:doc:
@@ -40,10 +40,56 @@ Time-stamp: <2019-04-03> 
      unable to recognize "/home/sylecn/sysadmin/de02-kubernetes/apps/get-client-ip.yaml": Get https://88.99.191.174:6443/api?timeout=32s: Forbidden port

    #+END_SRC



** 2019-09-03 how to run get-client-ip in dev env.

- without history api:

  stack exec get-client-ip

- with history api:

  SERVE_HISTORY_PAGE=1 stack exec get-client-ip



- to run stress test:

  ab -c 20 -n 100000 http://localhost:8081/



  curl http://localhost:8081/_calls



* later                                                               :entry:

* current                                                             :entry:

** 

** 2019-09-03 how to deploy to prod?

- test code locally.

- build docker image

  run ./build-docker-image.sh

- update yaml file and apply it.

  update image version in yaml.

  kubectl apply -f ~/sysadmin/de02-kubernetes/apps/get-client-ip.yaml



* done                                                                :entry:

** 2019-09-03 bug: when /_calls is not called, all IP records (thunks) are saved

in memory because of lazy evaluation. This cost a memory leak.



search: haskell memory leak because of lazy evaluation



I can use the variable once, so that it got evaluated.

for example, print it.



data is kept in MVar (rcCallHistory rc)



    , rcCallHistory :: MVar (S.Seq CallHistory)



How to rewrite this so no trunks are kept in the MVar:

      liftIO $ putMVar (rcCallHistory rc)

                       (boundedPushRight callHistory (rcHistorySize rc) newEntry)



check boundedPushRight definition. Maybe it's like foldl vs foldl'.



- in writeCallHistoryMaybe, add ! pattern for callHistory. problem solved.



      !callHistory <- liftIO $ takeMVar (rcCallHistory rc)



  This will evaluate callHistory, so thunks are gone at this step.



- DONE run stress test on the api locally to check the memory leak is solved

  when code fixed.



** 2019-04-03 v1.2.0 bug, should show last 10 calls. not all calls.

- add env var HISTORY_SIZE, default 10.

- dev