* COMMENT -*- mode: org -*- #+Date: 2019-04-02 Time-stamp: <2020-01-12> #+STARTUP: content * notes :entry: ** 2019-04-02 how to deploy get-client-ip? :doc: - update code as necessary. update version in get-client-ip.cabal - build project using stack. stack build --test --pedantic - test the app stack exec get-client-ip env SERVE_HISTORY_PAGE=1 stack exec get-client-ip env SERVE_HISTORY_PAGE=1 HISTORY_SIZE=3 stack exec get-client-ip curl http://127.0.0.1:8081/ - build docker image and push it to docker registry ./build-docker-image.sh - update docker image version in k8s app yaml ~/sysadmin/de02-kubernetes/apps/get-client-ip.yaml - deploy k8s app on de04, kaf ~/d/k8s/apps/get-client-ip.yaml # or on ryzen5, # usede04k8s # kaf ~/sysadmin/de02-kubernetes/apps/get-client-ip.yaml on ryzen5, usesh01k8s kaf ~/sysadmin/sh01-kubernetes/apps/get-client-ip.yaml - commit changes to git. project git. k8s app yaml git. ~/sysadmin/de02-kubernetes/ - problem - how to use proxy with kubectl apply? #+BEGIN_SRC sh sylecn@ryzen5:~/haskell/get-client-ip$ env https_proxy=127.0.0.1:8123 kubectl apply -f ~/sysadmin/de02-kubernetes/apps/get-client-ip.yaml unable to recognize "/home/sylecn/sysadmin/de02-kubernetes/apps/get-client-ip.yaml": Get https://88.99.191.174:6443/api?timeout=32s: Forbidden port unable to recognize "/home/sylecn/sysadmin/de02-kubernetes/apps/get-client-ip.yaml": Get https://88.99.191.174:6443/api?timeout=32s: Forbidden port unable to recognize "/home/sylecn/sysadmin/de02-kubernetes/apps/get-client-ip.yaml": Get https://88.99.191.174:6443/api?timeout=32s: Forbidden port sylecn@ryzen5:~/haskell/get-client-ip$ http_proxy=127.0.0.1:8123 kubectl apply -f ~/sysadmin/de02-kubernetes/apps/get-client-ip.yaml unable to recognize "/home/sylecn/sysadmin/de02-kubernetes/apps/get-client-ip.yaml": Get https://88.99.191.174:6443/api?timeout=32s: Forbidden port unable to recognize "/home/sylecn/sysadmin/de02-kubernetes/apps/get-client-ip.yaml": Get https://88.99.191.174:6443/api?timeout=32s: Forbidden port unable to recognize "/home/sylecn/sysadmin/de02-kubernetes/apps/get-client-ip.yaml": Get https://88.99.191.174:6443/api?timeout=32s: Forbidden port #+END_SRC ** 2019-09-03 how to deploy to prod? TLDR version :doc: - test code locally. - build docker image run ./build-docker-image.sh - update image version in yaml file and apply it. kubectl apply -f ~/sysadmin/de02-kubernetes/apps/get-client-ip.yaml ** 2019-09-03 how to run get-client-ip in dev env. - without history api: stack exec get-client-ip - with history api: SERVE_HISTORY_PAGE=1 stack exec get-client-ip - to run stress test: ab -c 20 -n 100000 http://localhost:8081/ curl http://localhost:8081/_calls ** 2020-01-12 APIs :doc: - GET / return client public IP address - GET /_calls return recent calls - GET /_calls/unique return recent calls from unqiue IPs. * later :entry: * current :entry: ** * done :entry: ** 2020-01-12 add an API to show recent unique IPs. - GET /_calls/unique return recent client unique IPs. only when history is enabled. - writeCallHistoryMaybe if this IP exists, drop that entry, insert new entry for it. if this IP doesn't exist, push new entry, remove the oldest entry if length exceeded max history size. - works on first try. very cool. ** 2019-09-03 bug: when /_calls is not called, all IP records (thunks) are saved in memory because of lazy evaluation. This cost a memory leak. search: haskell memory leak because of lazy evaluation I can use the variable once, so that it got evaluated. for example, print it. data is kept in MVar (rcCallHistory rc) , rcCallHistory :: MVar (S.Seq CallHistory) How to rewrite this so no trunks are kept in the MVar: liftIO $ putMVar (rcCallHistory rc) (boundedPushRight callHistory (rcHistorySize rc) newEntry) check boundedPushRight definition. Maybe it's like foldl vs foldl'. - in writeCallHistoryMaybe, add ! pattern for callHistory. problem solved. !callHistory <- liftIO $ takeMVar (rcCallHistory rc) This will evaluate callHistory, so thunks are gone at this step. - DONE run stress test on the api locally to check the memory leak is solved when code fixed. ** 2019-04-03 v1.2.0 bug, should show last 10 calls. not all calls. - add env var HISTORY_SIZE, default 10. - dev - how to make {-# ANN module "HLint: ignore Redundant do" #-} work with OverloadedStrings extension? {-# ANN module ("HLint: ignore Redundant do" :: String) #-} - fixed in v1.2.1 ** 2019-04-02 make get-client-ip build docker image and deploy on gocd. - uploading docker image from ryzen5 host is not good. oh, this image is public and on docker hub. see ~/sysadmin/de02-kubernetes/apps/get-client-ip.yaml it's using a public image. - ** 2019-04-02 show last 10 calls to get-client-ip. provide a web page that shows last 10 calls https://myip.emacsos.com/_calls | client-ip | time | user-agent | |--------------+----------------------------+-------------------| | 49.67.97.101 | 02/Apr/2019:13:58:24 +0800 | Python-urllib/2.7 | This allow me to get sheni's IP without login to de01 to see nginx log. - dev - a in RAM buffer will do. but there are multiple instances deployed on k8s. use k8s redis service then. - use an async action to insert record to redis. allow it to fail. if redis is not running, that action just do nothing except leave a error msg log. - I don't want to introduce redis to the app. maybe just save in ram. user can call GET /_calls multiple times to see results from all nodes. - which data type to use as the ring buffer? haskell - Purely functional (persistent) ring buffer - Stack Overflow https://stackoverflow.com/questions/52898190/purely-functional-persistent-ring-buffer Data.Sequence fixed length circular buffer in haskell - Stack Overflow https://stackoverflow.com/questions/6510175/fixed-length-circular-buffer-in-haskell vector: Efficient Arrays I will use Data.Sequence. - do I need to make it thread safe? is persistent data types safe by default? yes. because it always return a new Seq object. - put it in a MVar. - why import getCurrentTime fail? import Date.Time.Clock (getCurrentTime) I have time in pkg list. it's typo. "Date" vs "Data"!! - build html response blaze lucid Lucid: templating DSL for HTML https://chrisdone.com/posts/lucid/ - * wontfix :entry: