Skip to content
operational 30.3 KiB
Newer Older
Yuanle Song's avatar
Yuanle Song committed
* COMMENT -*- mode: org -*-
#+Date: 2019-11-12
Yuanle Song's avatar
Yuanle Song committed
Time-stamp: <2019-11-14>
Yuanle Song's avatar
Yuanle Song committed
#+STARTUP: content
* notes                                                               :entry:
** 2019-11-13 how to run it in dev env?
dotnet run -- --dry-run --itemize-changes --target d:\backup
dotnet run -- -n -i --target d:\backup
Yuanle Song's avatar
Yuanle Song committed
dotnet run -- -n --target d:\backup
Yuanle Song's avatar
Yuanle Song committed
try an ssh run:

dotnet run -- -n --target root@sylecn01.emacsos.com:/data/backup/PC-backup/B75I3/
it works.

file list works.
ssh transfer works.
remote logging works.

Yuanle Song's avatar
Yuanle Song committed
read target from config file works. now I can just run
dotnet run -- -i

** 2019-11-12 docs                                                :documents:
- rsync
  https://www.samba.org/ftp/rsync/rsync.html
- Basic Editing in Visual Studio Code
  https://code.visualstudio.com/docs/editor/codebasics

  Get Started with F# in Visual Studio Code | Microsoft Docs
  https://docs.microsoft.com/en-us/dotnet/fsharp/get-started/get-started-vscode
  install Ionide-fsharp extension.
- Argu
  http://fsprojects.github.io/Argu/
  Tutorial
  http://fsprojects.github.io/Argu/tutorial.html
  --help doesn't work, need error handler.
  Argu/Program.fs at master · fsprojects/Argu · GitHub
  https://github.com/fsprojects/Argu/blob/master/samples/Argu.Samples.LS/Program.fs
- F#
  F# Language Reference - F# | Microsoft Docs
  https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/
  F# Collection Types - F# | Microsoft Docs
  https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/fsharp-collection-types
  Modules - F# | Microsoft Docs
  https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/modules
  https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/namespaces
  fsharp-cheatsheet
  https://dungpa.github.io/fsharp-cheatsheet/
  Literals - F# | Microsoft Docs
  https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/literals
  [|' '|] is an array of character, length is 1.
  F# list is not the same as array. You can convert array to list via Array.toList
Yuanle Song's avatar
Yuanle Song committed
  String Class (System) | Microsoft Docs
  https://docs.microsoft.com/en-us/dotnet/api/system.string?view=netframework-4.8
  Environment.SpecialFolder
  https://docs.microsoft.com/en-us/dotnet/api/system.environment.specialfolder?view=netframework-4.8
  ApplicationData
  LocalApplicationData
  MyDocuments
  MyPictures
  DesktopDirectory
  Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData)
  DateTime.ToString Method (System) | Microsoft Docs
  https://docs.microsoft.com/en-us/dotnet/api/system.datetime.tostring?view=netframework-4.8
  Standard Date and Time Format Strings | Microsoft Docs
  https://docs.microsoft.com/en-us/dotnet/standard/base-types/standard-date-and-time-format-strings?view=netframework-4.8
Yuanle Song's avatar
Yuanle Song committed
  The Option type | F# for fun and profit
  https://fsharpforfunandprofit.com/posts/the-option-type/
  Pattern Matching - F# | Microsoft Docs
  https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/pattern-matching
Yuanle Song's avatar
Yuanle Song committed
- FSharpLint
  http://fsprojects.github.io/FSharpLint/
  http://fsprojects.github.io/FSharpLint/index.html
- FSharp.Configuration
  http://fsprojects.github.io/FSharp.Configuration/
  dotnet add package FSharp.Configuration --version 1.5.0
  http://fsprojects.github.io/FSharp.Configuration/IniTypeProvider.html
  I will use the ini provider.
  user don't need to escape string or pay attension to spaces in ini file.
  this lib doesn't work with dotnet core 3.
- Formatted text using printf | F# for fun and profit
  https://fsharpforfunandprofit.com/posts/printf/
- 

** 2019-11-13 install dir layout.
C:\Program Files\mbackup\rsync-w64\usr\bin\rsync.exe
C:\Program Files\mbackup\rsync-w64\usr\bin\ssh.exe
C:\Program Files\mbackup\publish\mbackup.exe
C:\Program Files\mbackup\publish\mbackup.dll
C:\ProgramData\mbackup\mbackup-default.exclude
C:\ProgramData\mbackup\mbackup-default.list
C:\ProgramData\mbackup\user-default.list
Yuanle Song's avatar
Yuanle Song committed
C:\ProgramData\mbackup\mbackup.txt

- optional user configs
  like /etc/mbackup/ in linux
  C:\ProgramData\mbackup\local.list
  C:\ProgramData\mbackup\local.exclude

  like ~/.mbackup/ in linux
  <My Documents>/mbackup/local.list
  <My Documents>/mbackup/local.exclude
** 2019-11-14 notes                                             :development:
- Argu optional param support.

  if option Target is optional, when try to get its value, you should use

  results.TryGetResult Target
  or
  results.GetResult(Target, defaultValue = xxx)
Yuanle Song's avatar
Yuanle Song committed
- WiX wxs file
  <Package InstallerVersion="300" Compressed="yes" Platform="x64"/>
  Platform="x64" has the same meaning as candle -arch x64.

- Wix for creating installer for 64bit app, you need to add in <Component> attribute
  Win64="yes"

  use candle -arch x64 can add Win64="yes" automatically if there is no Win64 attribute specified in Component.
  wix3.6 - WiX Heat tool, create Component with Win64 attribute - Stack Overflow
  https://stackoverflow.com/questions/11981498/wix-heat-tool-create-component-with-win64-attribute
- Wix to install file in another name, add Name attribute in <File>.
  <File Id="mbackup.exe" Name="mbackup.exe" Source="bin\Release\netcoreapp3.0\win10-x64\mbackup-for-windows.exe" KeyPath="yes" Checksum="yes"/>
Yuanle Song's avatar
Yuanle Song committed
- vscode powershell integrated console is not the same as powershell.
  it doesn't read $profile, and can't run ". $profile" command.
  it's best to press + to create a regular powershell terminal.
Yuanle Song's avatar
Yuanle Song committed
* later                                                               :entry:
** 2019-11-14 supports expand Downloads dir in user-default.list
Yuanle Song's avatar
Yuanle Song committed
** 2019-11-14 vscode f# doesn't support open a module
it can only support open a namespace.
using the vscode Ionide-fsharp extension.
Yuanle Song's avatar
Yuanle Song committed
* current                                                             :entry:
** 
** 2019-11-15 f# indent is difficult in vscode.
vscode should at least always indent using space for F#.

- try a code format tool.
  GitHub - fsprojects/fantomas: FSharp source code formatter
  https://github.com/fsprojects/fantomas
  dotnet tool install fantomas-tool -g

- there is context menu > format document.
  lonide-fsharp already support it.

  format document is on whole buffer. not just selected region.
  some code is not formatted properly.

  https://code.visualstudio.com/docs/editor/codebasics
  Formatting.
  Format Document (Ctrl+Shift+I) - Format the entire active file.
  Format Selection (Ctrl+K Ctrl+F) - Format the selected text.

  but there is not format selection in f# buffer.

Yuanle Song's avatar
Yuanle Song committed
** TODO 2019-11-15 additionally support <My Documents>/mbackup/local.list file.
This file is easier to open and backup.

no. programdata should be good enough.
but this is per user extra config file.
each user could have their own local file to backup.
one user don't want to manage all other user's local backup list.

yes. I should support this.

Both local.list and local.exclude.

** 2019-11-14 learn how to create an msi installer.
- WiX Toolset
  https://wixtoolset.org/
  c# - How to create MSI installer for ASP.NET Core application using Wix Toolset - Stack Overflow
  https://stackoverflow.com/questions/46878752/how-to-create-msi-installer-for-asp-net-core-application-using-wix-toolset

  Windows Installer XML (WiX)
  https://wixtoolset.org/documentation/manual/v3/main/

- Introducing dotGet: an installer for .NET Core CLI apps
  https://medium.com/@tonerdo/introducing-dotget-an-installer-for-net-core-cli-apps-9ba79ac7cd89

  requires nuget. then it can install your dotnet core console project.
  this is not what I need here.
- Packaging a self-contained .NET Core app for Windows Installer
  https://nblumhardt.com/2017/04/netcore-msi/

  This is the first time I’ve tried packaging a .NET Core app for the Windows
  Installer, so I thought I’d write these brief notes to help anyone else
  interested in taking the same path.

- should I include rsync-mingw64 in the same msi file?
  distributing it alone is not very useful, since people can just use msys2 or cygwin.

  If I include it in the same msi, I don't need to use wix bundle, and I can install rsync to the same dir as mbackup.
  it seems cleaner this way. Just like a VPN client bundle openvpn or a python program bundle python.
  yes. include it in the same msi and install in a sub dir.
- try Wix.
  - install wix.
    requires .net 3.5 windows feature.
  - Building Installation Package Bundles
    https://wixtoolset.org/documentation/manual/v3/bundle/
  - How To Guides
    https://wixtoolset.org/documentation/manual/v3/howtos/
  - How To: Add a File to Your Installer
    https://wixtoolset.org/documentation/manual/v3/howtos/files_and_registry/add_a_file.html
  - create mbackup.wxs
Yuanle Song's avatar
Yuanle Song committed

    build mbackup binaries
    dotnet publish -c Release
  - TODO how to show a message when installer is finished succesfully?
    currently it installs very fast and just exit without any user notification.
    maybe show a installer window and let user click Next and Finish.
  - TODO how to add some dir to PATH?
    I only have one exe. Maybe just create a start menu or desktop shortcut?
    create desktop shortcut to mbackup.exe
  - TODO how to support upgrade when I click a new msi?
    currently every msi is a standalone msi and will always install fresh.
    is the UpgradeCode used for this?
Yuanle Song's avatar
Yuanle Song committed

    does <Product> version have any effect on generated msi file?
    how to integrate this version with dotnet application version?
    *now*
  - mbackup.msi works on B75I3 host.
  - try mbackup.msi on win 10 VM.
    how to require dotnet core 3.0 in .wxs file?
    search: wix NetFxExtension symbol for .net core 3
    asp.net mvc - Wix package ID for .NET core runtime 1.0.3 - Stack Overflow
    https://stackoverflow.com/questions/42908913/wix-package-id-for-net-core-runtime-1-0-3
    requires dotnet core 3 runtime on target node.
    backup to local dir works.
    backup to remote ssh node...works.
    - problems
      - 2019-11-14T10:50:13 ERROR Read/write file failed: System.Private.CoreLib Could not find a part of the path 'C:\Users\IEUser\AppData\Local\mbackup\mbackup.list'.
        Does the dir exist? no. Create it at runtime.
        fixed.
      - backup to remote ssh node.
        Warning: Identity file /cygdrive/c/Users/IEUser/.ssh/id_rsa not accessible: No such file or directory.
        Can't open user config file /cygdrive/c/Users/IEUser/.ssh/config: No such file or directory
        DONE do not specify config file if it doesn't exist.
        DONE fail fast if ssh key doesn't exist. mbackup will only support key based auth.
        DONE ssh options should only be added when backup to remote node.

        it works.
        PS C:\Users\IEUser> &"C:\Program Files\mbackup\publish\mbackup.exe" --target root@sylecn01.emacsos.com:/data/backup/PC-backup/IEUser --ssh-key E:\id_rsa --remote-user sylecn
      - TODO failwith should not be used in UI code. It throws unhandled exception. System.Exception.
        search: f# exit early without shifting to the right
        in haskell, I use maybeT and eitherT etc.

        in f#, you should use exception.
Yuanle Song's avatar
Yuanle Song committed
- problems
  - each file require it's own <Component> tag.
    rsync mingw have many files. I seems I need to generate an installer for
    rsync for windows.
  - search: wix only install file if it does not exist
    windows installer - Copy if not exist in WiX - Stack Overflow
    https://stackoverflow.com/questions/1912037/copy-if-not-exist-in-wix
    installer - What is the wix 'KeyPath' attribute? - Stack Overflow
    https://stackoverflow.com/questions/2003043/what-is-the-wix-keypath-attribute/2003366#2003366
    Component Table - Win32 apps | Microsoft Docs
    https://docs.microsoft.com/en-us/windows/win32/msi/component-table?redirectedfrom=MSDN
    How To: Check for .NET Framework Versions
    https://wixtoolset.org/documentation/manual/v3/howtos/redistributables_and_install_checks/check_for_dotnet.html
  - WONTFIX how to unpack zip file to target dir?
Yuanle Song's avatar
Yuanle Song committed
    I don't want to create a Component for every file there.

    windows installer - Wix custom action to unzip a file - Stack Overflow
    https://stackoverflow.com/questions/47691477/wix-custom-action-to-unzip-a-file

    the suggestion is to include the files in msi.
    or unzip the files at app runtime.

    I will try create a msi file for mingw64 rsync.
    then use wix bundle to install it as dependency.

    - search: wix install all files in a folder

    - Include all Files in Bin folder in Wix installer - Stack Overflow
      https://stackoverflow.com/questions/36756311/include-all-files-in-bin-folder-in-wix-installer

      create rsync-mingw64-files.wxs
      &"C:\Program Files (x86)\WiX Toolset v3.11\bin\heat.exe" dir "D:\downloads\apps\rsync-w64" -cg RsyncHeatGenerated -dr ProgramFiles64Folder -var var.RsyncSourceDir -gg -nologo -out rsync-mingw64-files.wxs -sw5150

      compile and build rsync-mingw64.msi
      &"C:\Program Files (x86)\WiX Toolset v3.11\bin\candle.exe" -dRsyncSourceDir=D:\downloads\apps\rsync-w64\  .\rsync-mingw64.wxs .\rsync-mingw64-files.wxs
      &"C:\Program Files (x86)\WiX Toolset v3.11\bin\light.exe" .\rsync-mingw64.wixobj .\rsync-mingw64-files.wixobj -o rsync-mingw64.msi

      build is getting complex. I need make and Makefile.
    - problems
      - how to pass in preprocessor variable?
        WiX undefined preprocessor variable - Stack Overflow
        https://stackoverflow.com/questions/2063630/wix-undefined-preprocessor-variable
        To pass values for preprocessor variables like $(var.MyProject.TargetDir), use options like -dMyProject.TargetDir=c:\foo.
      - why rsync-mingw64 is installed to C:\Program Files (x86)\rsync-w64? I expect C:\Program Files\rsync-w64\
        wix-users - 64 Bit program files folder
        http://windows-installer-xml-wix-toolset.687559.n2.nabble.com/64-Bit-program-files-folder-td5948080.html
      - INVALID make pattern rule doesn't work in windows
        it works. it is because my source file is missing.
      - why mbackup.exe is 32 bit component?
        try this:
        dotnet publish -c Release -r win10-x64
        still same error.
        D:\sylecn_docs\projects\mbackup-for-windows\mbackup.wxs(16) : error LGHT0204 : ICE80: This 32BitComponent mbackup.exe uses 64BitDirectory APPLICATIONROOTDIRECTORY
        search: dotnet core how to build x64 exe file
        dotnet is building x64 exe file. it's WIX issue.
        need to add Win64="yes" in <Component> attribute.

  - DONE how to run wix to create msi file?
    two command line tool
    light
    candle
    it's not added in PATH.
    C:\Program Files (x86)\WiX Toolset v3.11\bin\light.exe
    C:\Program Files (x86)\WiX Toolset v3.11\bin\candle.exe

    There is offline doc.
    C:\Program Files (x86)\WiX Toolset v3.11\doc\WiX.chm
    C:\Program Files (x86)\WiX Toolset v3.11\doc\msi.chm

    do read the WiX Tutorial.
    https://www.firegiant.com/wix/tutorial/
    https://www.firegiant.com/wix/tutorial/getting-started/

    &"C:\Program Files (x86)\WiX Toolset v3.11\bin\candle.exe" .\mbackup.wxs
    &"C:\Program Files (x86)\WiX Toolset v3.11\bin\light.exe" .\mbackup.wixobj
    it works.
    mbackup.msi file is created.
  - the mbackup.exe can't run without all those dlls.
    dotnet publish -c Release -r win10-x64 --nologo
    maybe I should build a exe that requires dotnet core 3 to run.
    I don't want to create a big msi.
    try this:
    dotnet publish -c Release --nologo
    and update wix xml files.

    need to add all files in bin\Release\netcoreapp3.0\publish\ dir.
    use heat to create another wxs file.
  - heat can't add some dll.
    "C:\Program Files (x86)\WiX Toolset v3.11\bin\heat.exe" dir bin\Release\netcoreapp3.0\publish -cg MbackupHeatGenerated -dr MBACKUP_PROGRAM_FILES -var var.MbackupPublishDir -gg -nologo -out mbackup-files.wxs -sw5150
    heat.exe : warning HEAT5151 : Could not harvest data from a file that was expected to be an assembly: D:\sylecn_docs\projects\mbackup-for-windows\bin\Release\netcoreapp3.0\publish\Argu.dll. If this file is not an assembly you can ignore this warning. Otherwise, this error detail may be helpful to diagnose the failure: 未能加载文件或程序集“FSharp.Core, Version=4.4.3.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a”或它的某一个依赖项。系统找不到
    指定的文件。
    heat.exe : warning HEAT5151 : Could not harvest data from a file that was expected to be an assembly: D:\sylecn_docs\projects\mbackup-for-windows\bin\Release\netcoreapp3.0\publish\mbackup.dll. If this file is not an assembly 
    you can ignore this warning. Otherwise, this error detail may be helpful to diagnose the failure: 未能加载文件或程序集“System.Runtime, Version=4.2.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a”或它的某一个依赖项。系统
    找不到指定的文件。

    .net - Wix Toolkit: Heat balking about DLL's - Stack Overflow
    https://stackoverflow.com/questions/43992433/wix-toolkit-heat-balking-about-dlls
    com - Cannot register DLL using WiX - Stack Overflow
    https://stackoverflow.com/questions/12463256/cannot-register-dll-using-wix
  - TODO dup file?
    <Component Id="cmp981FC09307D7CB82695815FAAFCC646D" Directory="dir02A4AE56AE905AEEE7E2D4F4410A9748" Guid="{DB94490C-32CA-4B14-A96D-A95D4B557D01}">
                <File Id="fil432C46247A06CCDB891611045DD37081" KeyPath="yes" Source="$(var.MbackupPublishDir)\runtimes\win\lib\netstandard2.0\System.Security.Cryptography.ProtectedData.dll" />
    <Component Id="cmp5190411B4FE56A0E0CE782BE6C1B537E" Directory="dir1B0C588BD61288F160D459591DA01603" Guid="{F0613E24-B22B-4A58-A054-380BDF2B857C}">
                <File Id="filE0C94795A0B5F482BD659FBB774641C7" KeyPath="yes" Source="$(var.MbackupPublishDir)\System.Security.Cryptography.ProtectedData.dll" />
    why there are duplicated dll in publish dir? check publish doc.
    this is the only duplicated file. I will ignore it for now.
  - DONE ssh.exe path escape issue.
    &"C:\Program Files\mbackup\rsync-w64\usr\bin\rsync.exe" -h --stats -togr --delete --delete-excluded --ignore-missing-args --files-from=/cygdrive/c/Users/sylecn/AppData/Local/mbackup/mbackup.list --exclude-from=/cygdrive/c/ProgramData/mbackup/mbackup-default.exclude --log-file=/cygdrive/c/Users/sylecn/AppData/Local/mbackup/mbackup.log -e "/cygdrive/c/Program Files/mbackup/rsync-w64/usr/bin/ssh.exe -F /cygdrive/c/Users/sylecn/.ssh/config -i /cygdrive/c/Users/sylecn/.ssh/id_rsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" --remote-option=--log-file=/var/log/mbackup/B75I3.log --chown=sylecn:sylecn / root@sylecn01.emacsos.com:/data/backup/PC-backup/B75I3/
    rsync: Failed to exec /cygdrive/c/Program: No such file or directory (2)

    &"C:\Program Files\mbackup\rsync-w64\usr\bin\rsync.exe" -h --stats -togr --delete --delete-excluded --ignore-missing-args --files-from=/cygdrive/c/Users/sylecn/AppData/Local/mbackup/mbackup.list --exclude-from=/cygdrive/c/ProgramData/mbackup/mbackup-default.exclude --log-file=/cygdrive/c/Users/sylecn/AppData/Local/mbackup/mbackup.log -e "'/cygdrive/c/Program Files/mbackup/rsync-w64/usr/bin/ssh.exe' -F /cygdrive/c/Users/sylecn/.ssh/config -i /cygdrive/c/Users/sylecn/.ssh/id_rsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" --remote-option=--log-file=/var/log/mbackup/B75I3.log --chown=sylecn:sylecn / root@sylecn01.emacsos.com:/data/backup/PC-backup/B75I3/
Yuanle Song's avatar
Yuanle Song committed
** 2019-11-12 make code work in a specific dir. then create an installer.
- bundle dotnet core 3 with installer.
- install binary files to %programfiles%
- install mbackup config files to %programdata%
- install scheduled tasks as admin

  SCHTASKS /Create /?
  # run mbackup 15m after user logon.
Yuanle Song's avatar
Yuanle Song committed
  SCHTASKS /Create /NP /SC ONLOGON /TN mbackup-logon /TR "\"<path\to\mbackup.exe>\" \"args\"" /DELAY 15:00
  # run mbackup at 10am and 4pm.
Yuanle Song's avatar
Yuanle Song committed
  SCHTASKS /Create /NP /SC DAILY /TN mbackup-morning /TR "\"<path\to\mbackup.exe>\" \"args\"" /ST 10:00 /ET 13:00 /K
  SCHTASKS /Create /NP /SC DAILY /TN mbackup-afternoon /TR "\"<path\to\mbackup.exe>\" \"args\"" /ST 16:00 /ET 19:00 /K

  # debug purpose, one time only
Yuanle Song's avatar
Yuanle Song committed
  SCHTASKS /Create /NP /SC ONCE /TN mbackup-test /TR "\"<path\to\mbackup.exe>\" \"args\"" /ST 19:18
Yuanle Song's avatar
Yuanle Song committed
  In admin powershell,
  SCHTASKS /Create /NP /SC ONCE /TN mbackup-test /TR $mbackupexe /ST 19:32
  /NP option requires admin privilledge.
  it works.
Yuanle Song's avatar
Yuanle Song committed

Yuanle Song's avatar
Yuanle Song committed
  to delete it,
  schtasks /delete /tn mbackup-test
Yuanle Song's avatar
Yuanle Song committed

- config is installed/saved to %appdata% roaming dir.
  - problems
    - config should be system wide. %appdata% is per-user dir.
      where should I save global config file in windows 10?

      Windows uses the %APPDATA% directory for user specific application 
      configuration files. The %PROGRAMDATA% folder is used for system wide
      application configuration files, though it appears that Windows Store
      apps may not be able to use %PROGRAMDATA%.

      C:\ProgramData\mbackup\mbackup-default.list
- on B75I3, rsync is here.
  D:\downloads\apps\rsync-w64-3.1.3-2-standalone\usr\bin\rsync.exe
  
  scheduled task command is:
  <Actions Context="Author">
    <Exec>
      <Command>D:\downloads\apps\rsync-w64-3.1.3-2-standalone\usr\bin\rsync.exe</Command>
      <Arguments>--stats -togr --chown=sylecn:sylecn --exclude-from=/cygdrive/d/sylecn_docs/texts/configs/rsync-exclude --files-from=/cygdrive/d/sylecn_docs/texts/configs/rsync-file-list --log-file=/cygdrive/d/sylecn_docs/rsync-b75i3.log -e ".\ssh.exe -F c:/users/sylecn/.ssh/config -i c:/Users/sylecn/.ssh/id_rsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" / sheni:/data/backup/server-backup/b75i3/</Arguments>
      <WorkingDirectory>D:\downloads\apps\rsync-w64-3.1.3-2-standalone\usr\bin\</WorkingDirectory>
    </Exec>
  </Actions>
- I don't want to bundle python with mbackup for windows, so I will use
  C# and dotnet core 3 to create mbackup console executable.
  
  mbackup --cron
  # run the command as if run in scheduled task.

  mbackup
  # run backup command interactively.
  
  create new console project.
  dotnet new console -lang=F#
  
  Try F# this time. it's said dotnet have very good F# support.
  It's a small program, should be easy to handle.
- implementation
  - install-package argu
    can't find package.
    https://www.nuget.org/packages/Argu
    try this:
    dotnet add package Argu --version 5.5.0
    it works.
  - how to change output exe file name?
    can't find it. won't fix. default name is mbackup-for-windows.dll/exe
  - make rsync command work.
  - how to backup only documents in "My Documents" and "Downloads"?
    I don't want to backup exe/msi/zip/rar/7z/iso/tar.gz files in those dir.
    Only backup pdf/docx/xlsx etc.

    Maybe just let user add "My Documents" in local.list.
    I won't backup anything by default.
    This is not good enough. docs should be backed up by default.

    can I backup a zip file if I have Documents/**.zip in exclude list?
  - TODO F# doesn't have a free logging framework yet?
    logary/LICENSE.md at master · logary/logary · GitHub
    https://github.com/logary/logary/blob/master/LICENSE.md
    this one is not free software.
  - 

Yuanle Song's avatar
Yuanle Song committed
** 2019-11-12 how to test it? test it in a win 10 VM?
- search: lite weight win 10 VM

  There are pirated specialized versions of Windows 7 and XP out there called
  TinyXP and Tiny7. They are very stripped down and light weight.

  Tiny7 is created using AIK.
  https://en.wikipedia.org/wiki/Windows_Automated_Installation_Kit
- try get tiny7 iso/disk image.
  tiny7 is no longer maintained.

  try the old version.
  Tiny 7 Windows : eXPreience : Free Download, Borrow, and Streaming : Internet Archive
  https://archive.org/details/Tiny7
  2017.5
- I have a 4.1G MSEdge win10 vmdk disk on sheni. from 2016.
  maybe just use this.
- there is no enough RAM on B75I3. I will stop bogon VM and run win 10 VM.
- 

Yuanle Song's avatar
Yuanle Song committed
** 2019-11-12 mbackup for windows                                :featurereq:
Yuanle Song's avatar
Yuanle Song committed
- features
  - windows installer.
    bundles mingw rsync and ssh.
    they are NOT installed to PATH. so it will not affect other ssh programs.
  - backup file list;
    user file list;
    default exclude file list;
    user exclude file list;
    etc
    They have different value compared to mbackup for linux.
  - script designed to run via windows scheduled task.
    no service installed. no auto starts.

    taskschd.msc

- implementation.
  - there is rsync running on B75I3.

    exported tasksched.msc config as xml.
    that XML is not portable.
    it contains hostname.
  - 
* done                                                                :entry:
Yuanle Song's avatar
Yuanle Song committed
** 2019-11-13 next todos; scheduled task command line.
- DONE fix TODOs in F# code
- DONE add rsync command and arguments for running in windows.
  i.e. -e etc.

  <Arguments>--stats -togr --chown=sylecn:sylecn --exclude-from=/cygdrive/d/sylecn_docs/texts/configs/rsync-exclude
   --files-from=/cygdrive/d/sylecn_docs/texts/configs/rsync-file-list
    --log-file=/cygdrive/d/sylecn_docs/rsync-b75i3.log
     -e ".\ssh.exe -F c:/users/sylecn/.ssh/config -i c:/Users/sylecn/.ssh/id_rsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null"
      / sheni:/data/backup/server-backup/b75i3/</Arguments>

- DONE test param parsing is working.
  command line param > env var > mbackup default value.
  support --node-name param.
- DONE test run in console and scheduled task.
  run in console works.
  try run in scheduled task.

  SCHTASKS /Create /?
  # run mbackup 15m after user logon.
  SCHTASKS /Create /U <username> /SC ONLOGON /TN mbackup-logon /TR "\"<path\to\mbackup.exe>\" \"args\"" /DELAY 15:00
  # run mbackup at 10am and 4pm.
  SCHTASKS /Create /U <username> /SC DAILY /TN mbackup-morning /TR "\"<path\to\mbackup.exe>\" \"args\"" /ST 10:00 /ET 13:00 /K
  SCHTASKS /Create /U <username> /SC DAILY /TN mbackup-afternoon /TR "\"<path\to\mbackup.exe>\" \"args\"" /ST 16:00 /ET 19:00 /K

  # debug purpose, one time only
  SCHTASKS /Create /SC ONCE /TN mbackup-test /TR "\"<path\to\mbackup.exe>\" \"args\"" /ST 19:18

  in powershell,
  $mbackupexe = "D:\sylecn_docs\projects\mbackup-for-windows\bin\Debug\netcoreapp3.0\mbackup-for-windows.exe"
  SCHTASKS /Create /SC ONCE /TN mbackup-test /TR $mbackupexe /ST 19:21
  default run as user is current user.

  in admin powershell,
  SCHTASKS /Create /SC ONCE /TN mbackup-test /TR $mbackupexe /ST 19:32 /NP
  /NP option requires admin privilledge.
  it works.

  to delete it,
  schtasks /delete /tn mbackup-test

  - problems
    - how many scheduled task to run on a multi-user PC?
      each user have its own user-default.list expansion.
      should I iter over all users on PC?
      I think only current user can get it's profile dir and special dirs.
    - maybe config mbackup to run after user logon. with 15minute delay.
      always run as current user.
    - how to not require any param when running mbackup.exe?
      put TARGET and other option in a config file?
      define system level TARGET env variable.

      use config file is easier for user to edit and making the change effective.
      userConfigDir / mbackup.conf

      search: F# read config file
      FSharp.Configuration
      http://fsprojects.github.io/FSharp.Configuration/
    - FSharp.Configuration missing reference to System.Runtime.Caching
      search: how to reference System.Runtime.Caching for dotnet core project
      https://www.nuget.org/packages/System.Runtime.Caching/
      dotnet add package System.Runtime.Caching --version 4.6.0
      should I add 4.0.0? Can I use a higher version?

      still not compatible.
      search: use FSharp.Configuration with dotnet core 3
    - give up on FSharp.Configuration.
    - try this:
      FsConfig
      https://www.demystifyfp.com/FsConfig/
      AppSettings is only supported in V0.0.6 or below.
      try F# AppSettings directly.
      If nothing is easy to use, write my own parser.
      Support similar config format as python wells lib.
    - ConfigurationManager.AppSettings Property (System.Configuration) | Microsoft Docs
      https://docs.microsoft.com/en-us/dotnet/api/system.configuration.configurationmanager.appsettings?view=netframework-4.8
      where should I save the App.config xml file?
      search: what is app.config
      App.Config: Basics and Best Practices - SubMain Blog
      https://blog.submain.com/app-config-basics-best-practices/
      What is App.config in C#.NET? How to use it? - Stack Overflow
      https://stackoverflow.com/questions/13043530/what-is-app-config-in-c-net-how-to-use-it
      Okay. This is not what I want.
      This is for application configuration (rarely change), not for user configuration (can change any time).
      dotnet will create <yourapp>.exe.config from your App.config file.
    - how to use multiple files in F# dotnet core project?
      search: f# module and namespace
      search: f# module
      Modules - F# | Microsoft Docs
      https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/modules
    - SCHTASKS /Create /SC ONCE /TN mbackup-test /TR $mbackupexe /ST 19:21
      this runs, it will show a console window. how to not show console window?

      search: schedule tasks do not show console window

      In the “General” tab, under the “Security options” section, select the Run
      whether user is logged on or not option. (This is the option that will make
      the command window not to appear when the task runs automatically.)

      try /NP option.

- create an installer. The installer should add scheduled task on install and
  delete scheduled task on removal.

- problems
  - how to write unit test in F#?
    Unit testing F# in .NET Core with dotnet test and NUnit - .NET Core | Microsoft Docs
    https://docs.microsoft.com/en-us/dotnet/core/testing/unit-testing-fsharp-with-nunit
    mkdir mbackup-tests
    cd mbackup-tests
    dotnet new nunit -lang F#
    //wait.
    dotnet test

    dotnet add reference ../mbackup-for-windows.fsproj
    dotnet build  # this will build both ref project and test project.
    dotnet test
  - how to support short option names?
    --dry-run -n
  - 
Yuanle Song's avatar
Yuanle Song committed
** 2019-11-14 support mbackup.txt config.
config file will have wells config file format.
empty lines and comment lines are ignored.
space around = is ignored.
if there are quotes on value side, it's trimmed.

target=user@host:port/path/
# target = user@host:port/path/
abc.def = some value
abc.def.ghi = 1233.45
foo = yes

env variable will be
TARGET
ABC_DEF
ABC_DEF_GHI
FOO

in F# code,
conf = WellsConfig("C:\path\to\file.txt")
conf.GetStr("target")
conf.GetFloat("abc.def.ghi")
conf.GetBool("foo")

it works.

Yuanle Song's avatar
Yuanle Song committed
** 2019-11-13 extra user default list.
Documents is replaced by real path.
Downloads
Pictures

User should always use forward slash in backup list and exclude list files.
These path are supported:
C:\Foo\Bar
C:/Foo/Bar
Pictures/Saved Pictures
Documents/

Yuanle Song's avatar
Yuanle Song committed
** 2019-11-13 build mbackup.list file from file list and exclude lists.
Yuanle Song's avatar
Yuanle Song committed
* wontfix                                                             :entry: