Commit 23147468 authored by Yuanle Song's avatar Yuanle Song
Browse files

v1.2.1.0 bugfix test case should not rely on files not in git tree

parent 4ca6ae63

operational

+154 −5
Original line number Diff line number Diff line 
* COMMENT -*- mode: org -*-

#+Date: 2018-05-04

Time-stamp: <2024-04-06>

Time-stamp: <2024-04-07>

#+STARTUP: content

* notes                                                               :entry:

** 2022-03-14 project dir structure
@@ -555,6 +555,155 @@ only first character is in path key. 
* current                                                             :entry:

** 

** 2024-04-06 rd client: log should use local datetime not UTC time.

https://hackage.haskell.org/package/tinylog-0.15.0/docs/src/System.Logger.Settings.html#iso8601UTC



newtype DateFormat = DateFormat

    { display :: UnixTime -> ByteString

    }



instance IsString DateFormat where

    fromString = DateFormat . formatUnixTimeGMT . pack



-- | ISO 8601 date-time format.

iso8601UTC :: DateFormat

iso8601UTC = "%Y-%0m-%0dT%0H:%0M:%0SZ"



UnixTime

https://hackage.haskell.org/package/unix-time-0.4.5/docs/Data-UnixTime.html#t:UnixTime



setFormat :: Maybe DateFormat -> Settings -> Settings



- formatUnixTime :: Format -> UnixTime -> IO ByteString 

  formatUnixTimeGMT :: Format -> UnixTime -> ByteString



  type doesn't match. to get local time, it needs to read TZ info from env.



  display :: UnixTime -> ByteString

  I won't fit. it's a pure function. you can't read TZ info.



- I need to patch tinylog setFormat or related caller.



  or just switch to another log framework that is more flexible.



  search: haskell tinylog datetime doesn't support local timezone



  does RIO has native logging support? if so, I can switch to it.



  fast-logger is popular. just switch to it.



  which logger does rd-api use?



  infol rc $ sformat ("rd-api " % string) cliVersion

  errorl rc0 $ sformat

  warnl rc0 $ sformat



- problems

  - make it build on agem10.

    stack build fail with clang. which is installed on agem10.

    usually I use gcc.

    search: haskell stack ld.gold -fPIC



    Now that Stackage supports GHC 9.2, is it easy to build ARM executables on macOS? : r/haskell

    https://www.reddit.com/r/haskell/comments/tqzxy1/now_that_stackage_supports_ghc_92_is_it_easy_to/



    #+begin_quote

    $ cabal v2-build --ghc-options '-fllvm'

    $ stack build --ghc-options '-fllvm'

    $ ghc -0 main -fllvm main.hs

    #+end_quote

    stack build --ghc-options '-fllvm'

    still fail.



    try a newer lts.

    lts-20.26



    tinylog is no longer in this lts. why? build fail?

    add it to extra-deps.

    now fail with same error as lts-18.27.



    #+begin_quote

    /usr/bin/ld.gold: error: /home/sylecn/.stack/programs/x86_64-linux/ghc-tinfo6-9.2.8/lib/ghc-9.2.8/rts/libCffi.a(unix64.o): requires dynamic R_X86_64_PC32 reloc against 'abort' which may overflow at runtime; recompile with -fPIC

    #+end_quote



    #+begin_quote

    requires unsupported dynamic reloc 11; recompile with -fPIC

    #+end_quote

    search: stack build error requires unsupported dynamic reloc 11; recompile with -fPIC



    search: ghc-tinfo6 requires dynamic R_X86_64_PC32 reloc against 'abort' which may overflow at runtime; recompile with -fPIC



  - search: can I add -fPIC in ghc-options



    HaskellR - Programming R in Haskell

    https://tweag.github.io/HaskellR/docs/faq.html



    Link errors on Fedora 23 · Issue #257 · tweag/HaskellR · GitHub

    https://github.com/tweag/HaskellR/issues/257



    this is from 2016.



  - try use ld bfd instead of ld gold. install gcc.

    stack build still use ld.gold.



  - I think it's clang issue.



    stack build -v > t1.log 2>&1

    less t1.log



    remove clang

    stack build -v > t2.log 2>&1



    ghc: could not execute: clang

    rm -rf ~/.stack/programs/x86_64-linux/ghc-tinfo6-8.10.7  ~/.stack/programs/x86_64-linux/ghc-tinfo6-9.2.8/ ~/.stack/programs/x86_64-linux/*.installed



    is debian clang version too old? why clang fail?

    debian clang 14.0

    https://releases.llvm.org/download.html

    latest is 17.0.1



    after remove clang and remove installed ghc, now build can continue. no

    -fPIC error.



  - build still fail using gcc.

    #+begin_src sh

      basement             > [77 of 84] Compiling Basement.Imports

      basement             > [78 of 84] Compiling Basement.Environment

      basement             > [79 of 84] Compiling Basement.Alg.String

      basement             > [80 of 84] Compiling Basement.String

      basement             > [81 of 84] Compiling Basement.Terminal.ANSI

      basement             > [82 of 84] Compiling Basement.From

      basement             > [83 of 84] Compiling Basement.Block.Builder

      basement             > [84 of 84] Compiling Basement.String.Builder

      basement             > copy/register

      basement             > Installing library in /home/sylecn/.stack/snapshots/x86_64-linux-tinfo6/825ee73e95cffd5da2814fedfcf130f346ff6d98f03df43cf1ee3e812806e28b/8.10.7/lib/x86_64-linux-ghc-8.10.7/basement-0.0.12-CTZ777ail2hHOrDOpE5nxE

      basement             > Registering library for basement-0.0.12..

      Progress 32/127          



      Error: [S-7282]

	     Stack failed to execute the build plan.

       

	     While executing the build plan, Stack encountered the error:

       

	     [S-7011]

	     While building package double-conversion-2.0.2.0 (scroll up to its section to see the error) using:

	     /home/sylecn/.stack/setup-exe-cache/x86_64-linux-tinfo6/Cabal-simple_6HauvNHV_3.2.1.0_ghc-8.10.7 --verbose=1 --builddir=.stack-work/dist/x86_64-linux-tinfo6/ghc-8.10.7 build --ghc-options " -fdiagnostics-color=always"

	     Process exited with code: ExitFailure 1 

      sylecn@agem10:~/projects/reliable-download$ 

    #+end_src



  - double-conversion    > gcc: fatal error: cannot execute ‘cc1plus’: execvp: No such file or directory

    setup g++



    yes. it works.

    so stack ghc only builds with gcc and g++. not clang. at least not clang 14.



    linux · Wiki · Glasgow Haskell Compiler / GHC · GitLab

    https://gitlab.haskell.org/ghc/ghc/-/wikis/building/preparation/linux

    tools · Wiki · Glasgow Haskell Compiler / GHC · GitLab

    https://gitlab.haskell.org/ghc/ghc/-/wikis/building/preparation/tools



    build GHC requires gcc and g++.



** 2024-04-06 rd client: when server side doesn't support GET /rd/ api.

give a more clear msg to client side.

it's not client side's fault.
@@ -599,10 +748,6 @@ IO () -> IO () 


I should not need to write showProgressLoop explicitly.



** 2022-03-15 TODO stack test should not rely on

/home/sylecn/persist/cache/ideaIC-2018.1.tar.gz



try use a smaller file within git tree.

** 2019-02-28 bug: rd-api -d java/

option -d: cannot parse value `java/'
@@ -622,6 +767,10 @@ policy in ovs can do it. 
  see stretch01 daylog.



* done                                                                :entry:

** 2022-03-15 stack test should not rely on

/home/sylecn/persist/cache/ideaIC-2018.1.tar.gz



try use a smaller file within git tree.

** 2018-05-10 use a proper module hierarchy.

import RD.Utils

import RD.Api.Lib

package.yaml

+2 −2
Original line number Diff line number Diff line 
name:                reliable-download

version:             1.2.0.0

version:             1.2.1.0

synopsis:            provide reliable download service via HTTP

description:         reliable-download web application and cli tool

homepage:            "https://gitlab.emacsos.com/sylecn/reliable-download"
@@ -7,7 +7,7 @@ bug-reports: "https://gitlab.emacsos.com/sylecn/reliable-download/issues 
license:             GPL-3

author:              Yuanle Song

maintainer:          sylecn@gmail.com

copyright:           "Copyright: (c) 2018, 2019, 2022 Yuanle Song"

copyright:           "Copyright: (c) 2018, 2019, 2022, 2024 Yuanle Song"

category:            Utilities

extra-source-files:

- README.md

test/TestApi.hs

+10 −7
Original line number Diff line number Diff line
@@ -108,17 +108,20 @@ spec = do 


  describe "fileRange, hGet n bytes" $ do

    it "should work" $ do

      contentLB <- liftIO $ fileRange "/home/sylecn/persist/cache/ideaIC-2018.1.tar.gz" 0 2097151

      let contentLength = 2400

      contentLB <- liftIO $ fileRange "./test/sha1sumFileRange1.dat" 0 contentLength

      let first10Byte = LB.take 10 contentLB

      sha1sumOnBytes first10Byte `shouldBe` "04bdadb7bd09681ea2d3f84210feff6549b6ab45"

      LB.length contentLB `shouldBe` 2097152

      let last10Byte = LB.drop (2097152 - 10) contentLB

      sha1sumOnBytes last10Byte `shouldBe` "c643bc36514fce49013d31f80775dbbc9bf9e9a7"

      sha1sumOnBytes first10Byte `shouldBe` "2059f97d0abc77d255109b52e5240d268225149f"

      fromIntegral (LB.length contentLB) `shouldBe` contentLength

      let last10Byte = LB.drop (fromIntegral (contentLength - 10)) contentLB

      sha1sumOnBytes last10Byte `shouldBe` "d079691750a673076a7d0b5ffaf5371c9981e868"



  describe "sha1sumFileRange" $ do

    it "should work" $ do

      sha1 <- liftIO $ sha1sumFileRange "/home/sylecn/persist/cache/ideaIC-2018.1.tar.gz" 0 2097151

      sha1 `shouldBe` "4690b050834d4059d40ad6f63bf91d6a4558bb71"

      -- to get expected hash,

      -- head -c 6 ./test/sha1sumFileRange1.dat |sha1sum

      sha1 <- liftIO $ sha1sumFileRange "./test/sha1sumFileRange1.dat" 0 5

      sha1 `shouldBe` "1f8ac10f23c5b5bc1167bda84b833e5c057a77d2"



-- | like get, but accept path in [T.Text] format and do url safe encoding.

getPath :: [T.Text] -> WaiSession st SResponse

test/sha1sumFileRange1.dat

0 → 100644
+320 −0
Original line number Diff line number Diff line 
abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz

abcdefg

hijklmn

opqrstu

vwxyz