operational

* COMMENT -*- mode: org -*-
#+Date: 2019-11-12
Time-stamp: <2019-11-17>
#+STARTUP: content
* notes                                                               :entry:
** 2019-11-13 how to run it in dev env?
dotnet run -- --dry-run --itemize-changes --target d:\backup
dotnet run -- -n -i --target d:\backup
dotnet run -- -n --target d:\backup

try an ssh run:

dotnet run -- -n --target root@sylecn01.emacsos.com:/data/backup/PC-backup/B75I3/
it works.

file list works.
ssh transfer works.
remote logging works.

read target from config file works. now I can just run
dotnet run -- -i

** 2019-11-12 docs                                                :documents:
- rsync
  https://www.samba.org/ftp/rsync/rsync.html
- Basic Editing in Visual Studio Code
  https://code.visualstudio.com/docs/editor/codebasics

  Get Started with F# in Visual Studio Code | Microsoft Docs
  https://docs.microsoft.com/en-us/dotnet/fsharp/get-started/get-started-vscode
  install Ionide-fsharp extension.
- Argu
  http://fsprojects.github.io/Argu/
  Tutorial
  http://fsprojects.github.io/Argu/tutorial.html
  --help doesn't work, need error handler.
  Argu/Program.fs at master · fsprojects/Argu · GitHub
  https://github.com/fsprojects/Argu/blob/master/samples/Argu.Samples.LS/Program.fs
- F#
  F# Language Reference - F# | Microsoft Docs
  https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/
  F# Collection Types - F# | Microsoft Docs
  https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/fsharp-collection-types
  Modules - F# | Microsoft Docs
  https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/modules
  https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/namespaces
  fsharp-cheatsheet
  https://dungpa.github.io/fsharp-cheatsheet/
  Literals - F# | Microsoft Docs
  https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/literals
  [|' '|] is an array of character, length is 1.
  F# list is not the same as array. You can convert array to list via Array.toList
  String Class (System) | Microsoft Docs
  https://docs.microsoft.com/en-us/dotnet/api/system.string?view=netframework-4.8
  Environment.SpecialFolder
  https://docs.microsoft.com/en-us/dotnet/api/system.environment.specialfolder?view=netframework-4.8
  ApplicationData
  LocalApplicationData
  MyDocuments
  MyPictures
  DesktopDirectory
  Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData)
  DateTime.ToString Method (System) | Microsoft Docs
  https://docs.microsoft.com/en-us/dotnet/api/system.datetime.tostring?view=netframework-4.8
  Standard Date and Time Format Strings | Microsoft Docs
  https://docs.microsoft.com/en-us/dotnet/standard/base-types/standard-date-and-time-format-strings?view=netframework-4.8
  The Option type | F# for fun and profit
  https://fsharpforfunandprofit.com/posts/the-option-type/
  Pattern Matching - F# | Microsoft Docs
  https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/pattern-matching
- FSharpLint
  http://fsprojects.github.io/FSharpLint/
  http://fsprojects.github.io/FSharpLint/index.html
- FSharp.Configuration
  http://fsprojects.github.io/FSharp.Configuration/
  dotnet add package FSharp.Configuration --version 1.5.0
  http://fsprojects.github.io/FSharp.Configuration/IniTypeProvider.html
  I will use the ini provider.
  user don't need to escape string or pay attension to spaces in ini file.
  this lib doesn't work with dotnet core 3.
- Formatted text using printf | F# for fun and profit
  https://fsharpforfunandprofit.com/posts/printf/
- Wix: a tool to create msi file
  WiX Toolset
  https://wixtoolset.org/
  Windows Installer XML (WiX)
  https://wixtoolset.org/documentation/manual/v3/main/

  To learn the basics, do read the WiX Tutorial.
  https://www.firegiant.com/wix/tutorial/
  https://www.firegiant.com/wix/tutorial/getting-started/

  Then you can follow the how-tos.
  How To Guides
  https://wixtoolset.org/documentation/manual/v3/howtos/

  How To: Implement a Major Upgrade In Your Installer
  https://wixtoolset.org/documentation/manual/v3/howtos/updates/major_upgrade.html

  Shortcut Element
  https://wixtoolset.org/documentation/manual/v3/xsd/wix/shortcut.html
  xml - Wix - ICE60 and ICE69 warnings - Stack Overflow
  https://stackoverflow.com/questions/21320334/wix-ice60-and-ice69-warnings/31192979
  From MSI to WiX, Part 10 – Shortcuts – Alex Shevchuk
  https://blogs.technet.microsoft.com/alexshev/2008/02/20/from-msi-to-wix-part-10-shortcuts/
  ICE errors
  https://docs.microsoft.com/en-us/windows/win32/msi/ice90
  https://docs.microsoft.com/en-us/windows/win32/msi/ice69

  CustomAction Element
  https://wixtoolset.org/documentation/manual/v3/xsd/wix/customaction.html

** 2019-11-13 install dir layout.
C:\Program Files\mbackup\rsync-w64\usr\bin\rsync.exe
C:\Program Files\mbackup\rsync-w64\usr\bin\ssh.exe
C:\Program Files\mbackup\publish\mbackup.exe
C:\Program Files\mbackup\publish\mbackup.dll

C:\ProgramData\mbackup\default-exclude.txt
C:\ProgramData\mbackup\default-list.txt
C:\ProgramData\mbackup\user-default-list.txt
C:\ProgramData\mbackup\mbackup-config.txt

C:\ProgramData\mbackup\local-list.txt
C:\ProgramData\mbackup\local-exclude.txt

- optional user configs
  C:\ProgramData\mbackup\local-list.txt
  C:\ProgramData\mbackup\local-exclude.txt

  - auto create these two files if they do not exist on target node.
    never overwrite existing file.

  - add shortcut to 3 files in start menu dir.
    mbackup/mbackup.exe
    mbackup/local-list.txt
    mbackup/local-exclude.txt
    mbackup/mbackup-config.txt

** 2019-11-14 notes                                             :development:
- Argu optional param support.

  if option Target is optional, when try to get its value, you should use

  results.TryGetResult Target
  or
  results.GetResult(Target, defaultValue = xxx)
- WiX wxs file
  <Package InstallerVersion="300" Compressed="yes" Platform="x64"/>
  Platform="x64" has the same meaning as candle -arch x64.

- Wix for creating installer for 64bit app, you need to add in <Component> attribute
  Win64="yes"

  use candle -arch x64 can add Win64="yes" automatically if there is no Win64 attribute specified in Component.
  wix3.6 - WiX Heat tool, create Component with Win64 attribute - Stack Overflow
  https://stackoverflow.com/questions/11981498/wix-heat-tool-create-component-with-win64-attribute
- Wix to install file in another name, add Name attribute in <File>.
  <File Id="mbackup.exe" Name="mbackup.exe" Source="bin\Release\netcoreapp3.0\win10-x64\mbackup-for-windows.exe" KeyPath="yes" Checksum="yes"/>
- vscode powershell integrated console is not the same as powershell.
  it doesn't read $profile, and can't run ". $profile" command.
  it's best to press + to create a regular powershell terminal.

* later                                                               :entry:
** TODO 2019-11-15 additionally support <My Documents>/mbackup/local-list.txt file.
This file is easier to open and backup.

no. programdata should be good enough.
but this is per user extra config file.
each user could have their own local file to backup.
one user don't want to manage all other user's local backup list.

yes. I should support this.

Both local.list and local.exclude.

- 2019-11-16 local.list is difficult to open in windows.
  use txt file instead for all file lists and ignore lists.

  like ~/.mbackup/ in linux
  <My Documents>/mbackup/local-list.txt
  <My Documents>/mbackup/local-exclude.txt

  If <My Documents>/mbackup/local-*.txt exists, it's merged to system level
  local-*.txt.

** 2019-11-14 supports expand Downloads dir in user-default.list
** 2019-11-14 vscode f# doesn't support open a module
it can only support open a namespace.
using the vscode Ionide-fsharp extension.
** 2019-11-16 ssh.exe Could not create directory '/home/IEUser/.ssh' error.
when rsync.exe or ssh.exe is run from cmd.exe, it will always show this error.
HOME env var is defined properly.

- create /etc/nsswitch.conf with the following content didn't fix it.
  db_home: windows

- openssh get user's home dir from getpwuid call.
  Check how ssh get home dir. maybe from /etc/passwd and similar syscall.
  check ssh-keygen source code.
  https://github.com/openssh/openssh-portable/blob/master/ssh-keygen.c
  pw = getpwuid(getuid());
  https://manpages.ubuntu.com/manpages/bionic/en/man3/getpwuid.3.html
  indeed it read config for passwd entry.
  how to bundle passwd file in mingw application?
** 2019-11-15 f# indent is difficult in vscode.
vscode should at least always indent using space for F#.

- try a code format tool.
  GitHub - fsprojects/fantomas: FSharp source code formatter
  https://github.com/fsprojects/fantomas
  dotnet tool install fantomas-tool -g

- there is context menu > format document.
  lonide-fsharp already support it.

  format document is on whole buffer. not just selected region.
  some code is not formatted properly.

  https://code.visualstudio.com/docs/editor/codebasics
  Formatting.
  Format Document (Ctrl+Shift+I) - Format the entire active file.
  Format Selection (Ctrl+K Ctrl+F) - Format the selected text.

  but there is not format selection in f# buffer.

* current                                                             :entry:
** 
** 2019-11-16 how to sign my msi executable?
so it doesn't show publisher unknown.

** 2019-11-12 make code work in a specific dir. then create an installer.
- bundle dotnet core 3 with installer.
- DONE install binary files to %programfiles%
- DONE install mbackup config files to %programdata%
- install scheduled tasks as admin

  SCHTASKS /Create /?
  # run mbackup 15m after user logon.
  SCHTASKS /Create /NP /SC ONLOGON /TN mbackup-logon /TR "\"<path\to\mbackup.exe>\" \"args\"" /DELAY 0015:00
  # run mbackup at 10am and 4pm.
  SCHTASKS /Create /NP /SC DAILY /TN mbackup-morning /TR "\"<path\to\mbackup.exe>\" \"args\"" /ST 10:00 /ET 14:00 /K
  SCHTASKS /Create /NP /SC DAILY /TN mbackup-afternoon /TR "\"<path\to\mbackup.exe>\" \"args\"" /ST 16:00 /ET 19:00 /K

  # debug purpose, one time only
  SCHTASKS /Create /NP /SC ONCE /TN mbackup-test /TR "\"<path\to\mbackup.exe>\" \"args\"" /ST 19:18

  In admin powershell,
  SCHTASKS /Create /NP /SC ONCE /TN mbackup-test /TR $mbackupexe /ST 19:32
  /NP option requires admin privilledge.
  it works.

  to delete it,
  schtasks /delete /tn mbackup-test

- config is installed/saved to %appdata% roaming dir.
  - problems
    - config should be system wide. %appdata% is per-user dir.
      where should I save global config file in windows 10?

      Windows uses the %APPDATA% directory for user specific application 
      configuration files. The %PROGRAMDATA% folder is used for system wide
      application configuration files, though it appears that Windows Store
      apps may not be able to use %PROGRAMDATA%.

      C:\ProgramData\mbackup\mbackup-default.list
- on B75I3, rsync is here.
  D:\downloads\apps\rsync-w64-3.1.3-2-standalone\usr\bin\rsync.exe
  
  scheduled task command is:
  <Actions Context="Author">
    <Exec>
      <Command>D:\downloads\apps\rsync-w64-3.1.3-2-standalone\usr\bin\rsync.exe</Command>
      <Arguments>--stats -togr --chown=sylecn:sylecn --exclude-from=/cygdrive/d/sylecn_docs/texts/configs/rsync-exclude --files-from=/cygdrive/d/sylecn_docs/texts/configs/rsync-file-list --log-file=/cygdrive/d/sylecn_docs/rsync-b75i3.log -e ".\ssh.exe -F c:/users/sylecn/.ssh/config -i c:/Users/sylecn/.ssh/id_rsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" / sheni:/data/backup/server-backup/b75i3/</Arguments>
      <WorkingDirectory>D:\downloads\apps\rsync-w64-3.1.3-2-standalone\usr\bin\</WorkingDirectory>
    </Exec>
  </Actions>
- I don't want to bundle python with mbackup for windows, so I will use
  C# and dotnet core 3 to create mbackup console executable.
  
  mbackup --cron
  # run the command as if run in scheduled task.

  mbackup
  # run backup command interactively.
  
  create new console project.
  dotnet new console -lang=F#
  
  Try F# this time. it's said dotnet have very good F# support.
  It's a small program, should be easy to handle.
- implementation
  - install-package argu
    can't find package.
    https://www.nuget.org/packages/Argu
    try this:
    dotnet add package Argu --version 5.5.0
    it works.
  - how to change output exe file name?
    can't find it. won't fix. default name is mbackup-for-windows.dll/exe
  - make rsync command work.
  - how to backup only documents in "My Documents" and "Downloads"?
    I don't want to backup exe/msi/zip/rar/7z/iso/tar.gz files in those dir.
    Only backup pdf/docx/xlsx etc.

    Maybe just let user add "My Documents" in local.list.
    I won't backup anything by default.
    This is not good enough. docs should be backed up by default.

    can I backup a zip file if I have Documents/**.zip in exclude list?
  - TODO F# doesn't have a free logging framework yet?
    logary/LICENSE.md at master · logary/logary · GitHub
    https://github.com/logary/logary/blob/master/LICENSE.md
    this one is not free software.
  - 2019-11-16 create scheduled task fail when installing.
    how to check the output of the command?
    msiexec /l*v log.txt /i mbackup-0.4.0.0.msi
    only log error
    msiexec /le loge.txt /i mbackup-0.4.0.0.msi

    it's mbackup.exe path problem.
    [INSTALLFOLDER] expands to empty string.

    [INSTALLFOLDER]\publish\mbackup.exe
    -->
    [MBACKUP_PROGRAM_FILES]publish\mbackup.exe
  - after this change, command runs fine in admin cmd.exe window.
    but when run in installer, still fail.
  - &quot;[SystemFolder]SCHTASKS.EXE&quot;
    -->
    SCHTASKS.EXE
    same error. not escaping issue.
  - error 1722 from simplest of .exe custom actions - Community
    https://community.flexera.com/t5/InstallShield-Forum/error-1722-from-simplest-of-exe-custom-actions/td-p/66380

    notepad.exe works in my case.
    when I exit notepad.exe, installer exit successfully.
    does it run as admin user? check.

    it's run as system user. not administrator or current user.
    it can't access desktop/documents etc.
    maybe that's the reason?

    » Using Windows Installer (WIX) to schedule a task as the system user RainbowInWater
    http://kamivaniea.com/?p=632

    SCHTASKS.EXE /Create /RU SYSTEM /RP /NP /SC ONLOGON /TN mbackup-logon /TR "C:\Program Files\mbackup\publish\mbackup.exe" /DELAY 0015:00

  - try add /RU SYSTEM /RP
    it works.

    TODO task will run as SYSTEM user though.
    how would home dir expand in SYSTEM user?
    I'd like task to run as currently logon user. for ONLOGON.

  - TODO why start menu shortcuts doesn't appear in win 10 guest?
    start menu folder is created. but only one exe there.
    txt file shortcuts are not added.

  - DAILY trigger, why repeat every 10min?

    #+BEGIN_QUOTE
    /RI   interval     Specifies the repetition interval in minutes. This is
                       not applicable for schedule types: MINUTE, HOURLY,
                       ONSTART, ONLOGON, ONIDLE, ONEVENT.
                       Valid range: 1 - 599940 minutes.
                       If either /ET or /DU is specified, then it defaults to
                       10 minutes.
    #+END_QUOTE
    just remove /ET? /K requires /ET.

    I can make use of /RI, run mbackup every 30min.
    TODO what happen if last run have not finished yet?
    does /RI interval count from last exit?

  - DONE mbackup start menu dir is not deleted when mbackup is uninstalled.
    // it is auto deleted now. tested 0.4.6.0 on B75I3.

    I want to install as per machine, not per user.
    changed reg root key to HKLM.

    c# - How can I create a shortcut on the Users Desktop folder - Stack Overflow
    https://stackoverflow.com/questions/29255922/how-can-i-create-a-shortcut-on-the-users-desktop-folder/29498210#29498210
    installer - How to create desktop shortcut for all users in WIX - Stack Overflow
    https://stackoverflow.com/questions/2211218/how-to-create-desktop-shortcut-for-all-users-in-wix

    per machine requires set ALLUSERS=1.
    how to set it?
    add inside <Product>
    <Property Id="ALLUSERS">1</Property>
    that result in error.

    Setting the 'AllUsers' option on Wix installer does not work - Stack Overflow
    https://stackoverflow.com/questions/17878460/setting-the-allusers-option-on-wix-installer-does-not-work
    set InstallScope="perMachine" should set ALLUSERS key.

    search: wix can't use HKLM InstallScope="perMachine"

  - info: install for all users is trouble some in wix.
    it's not clear how to install shortcuts.

    when install as per User. current user can edit mbackup-config.txt file.
    when install as per machine, only admin can edit that file.

    maybe just install per User.
    when install scheduled task, run as current user.
    task name should also include current user name.

  - 

** 2019-11-12 how to test it? test it in a win 10 VM?
- search: lite weight win 10 VM

  There are pirated specialized versions of Windows 7 and XP out there called
  TinyXP and Tiny7. They are very stripped down and light weight.

  Tiny7 is created using AIK.
  https://en.wikipedia.org/wiki/Windows_Automated_Installation_Kit
- try get tiny7 iso/disk image.
  tiny7 is no longer maintained.

  try the old version.
  Tiny 7 Windows : eXPreience : Free Download, Borrow, and Streaming : Internet Archive
  https://archive.org/details/Tiny7
  2017.5
- I have a 4.1G MSEdge win10 vmdk disk on sheni. from 2016.
  maybe just use this.
- there is no enough RAM on B75I3. I will stop bogon VM and run win 10 VM.
- 

** 2019-11-12 mbackup for windows                                :featurereq:
- features
  - windows installer.
    bundles mingw rsync and ssh.
    they are NOT installed to PATH. so it will not affect other ssh programs.
  - backup file list;
    user file list;
    default exclude file list;
    user exclude file list;
    etc
    They have different value compared to mbackup for linux.
  - script designed to run via windows scheduled task.
    no service installed. no auto starts.

    taskschd.msc

- implementation.
  - there is rsync running on B75I3.

    exported tasksched.msc config as xml.
    that XML is not portable.
    it contains hostname.
  - 
* done                                                                :entry:
** 2019-11-16 local-exclude.txt should convert win path to mingw path.
foo/bar
C:\foo\bar
/cygdrive/c/foo/bar

remove comments and empty lines.
if a line starts with [a-z]:, convert it to mingw path.

** 2019-11-14 learn how to create an msi installer.
- WiX Toolset
  https://wixtoolset.org/
  c# - How to create MSI installer for ASP.NET Core application using Wix Toolset - Stack Overflow
  https://stackoverflow.com/questions/46878752/how-to-create-msi-installer-for-asp-net-core-application-using-wix-toolset

  Windows Installer XML (WiX)
  https://wixtoolset.org/documentation/manual/v3/main/

- Introducing dotGet: an installer for .NET Core CLI apps
  https://medium.com/@tonerdo/introducing-dotget-an-installer-for-net-core-cli-apps-9ba79ac7cd89

  requires nuget. then it can install your dotnet core console project.
  this is not what I need here.
- Packaging a self-contained .NET Core app for Windows Installer
  https://nblumhardt.com/2017/04/netcore-msi/

  This is the first time I’ve tried packaging a .NET Core app for the Windows
  Installer, so I thought I’d write these brief notes to help anyone else
  interested in taking the same path.

- should I include rsync-mingw64 in the same msi file?
  distributing it alone is not very useful, since people can just use msys2 or cygwin.

  If I include it in the same msi, I don't need to use wix bundle, and I can install rsync to the same dir as mbackup.
  it seems cleaner this way. Just like a VPN client bundle openvpn or a python program bundle python.
  yes. include it in the same msi and install in a sub dir.
- try Wix.
  - install wix.
    requires .net 3.5 windows feature.
  - Building Installation Package Bundles
    https://wixtoolset.org/documentation/manual/v3/bundle/
  - How To Guides
    https://wixtoolset.org/documentation/manual/v3/howtos/
  - How To: Add a File to Your Installer
    https://wixtoolset.org/documentation/manual/v3/howtos/files_and_registry/add_a_file.html
  - create mbackup.wxs

    build mbackup binaries
    dotnet publish -c Release
  - TODO how to show a message when installer is finished succesfully?
    currently it installs very fast and just exit without any user notification.
    maybe show a installer window and let user click Next and Finish.
  - TODO how to add some dir to PATH?
    I only have one exe. Maybe just create a start menu or desktop shortcut?
    DONE create desktop shortcut to mbackup.exe
    installer - Create shortcut to desktop using WiX - Stack Overflow
    https://stackoverflow.com/questions/11868499/create-shortcut-to-desktop-using-wix
    it works.
    install will install two shortcuts.
    uninstall will remove both shortcuts.
  - DONE how to support upgrade when I click a new msi?
    currently every msi is a standalone msi and will always install fresh.
    is the UpgradeCode used for this?

    does <Product> version have any effect on generated msi file?
    how to integrate this version with dotnet application version?

    search: wix msi how to support upgrade
    How To: Implement a Major Upgrade In Your Installer
    https://wixtoolset.org/documentation/manual/v3/howtos/updates/major_upgrade.html
    - how to support L10N for wxs file? DowngradeErrorMessage
      <MajorUpgrade AllowDowngrades="no" DowngradeErrorMessage="!(loc.NewerVersionInstalled)"
                    AllowSameVersionUpgrades="no" />
    - in my application, downgrade should be supported.
    - note: <MajorUpgrade AllowDowngrades="yes"/> element should come after <Media>.
    - add --version support.
      search: dotnet application where to put version info and support --version parameter

      https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-build
      Build the project and set version 1.2.3.4 as a build parameter using the -p MSBuild option:
      dotnet build -p:Version=1.2.3.4

      how to get the Version in code?
      Get App Version in .NET Core - Edi.Wang
      https://edi.wang/post/2018/9/27/get-app-version-net-core
      First, let's see a typical .NET Core project file with versioning. We can define
      the version numbers for AssemblyVersion, FileVersion and Version in the project
      file (.csproj).

      <Project Sdk="Microsoft.NET.Sdk">
        <PropertyGroup>
          <OutputType>Exe</OutputType>
          <TargetFramework>netcoreapp2.1</TargetFramework>
          <AssemblyVersion>1.1.1.1</AssemblyVersion>
          <FileVersion>2.2.2.2</FileVersion>
          <Version>3.3.3.3-xyz</Version>
        </PropertyGroup>
      </Project>

    - problems
      - Could not create directory '/home/IEUser/.ssh'.
        how to make this error gone?
        check ssh option. maybe just set HOME env variable if it doesn't exist when starting rsync process?

        search: dotnet set environment variable
        Environment.SetEnvironmentVariable Method
      - how to get Version for use in Makefile?
        $(file < mbackup.fsproj) doesn't get expanded?
        try use $(shell) instead.
        FSPROJ := $(shell powershell -NoProfile -Command {cat mbackup.fsproj})
        still not working.
        FSPROJ has value of cat mbackup.fsproj
        search: makefile read file in windows
        no result.
        Make is not well supported in windows.

        FSPROJ := $(shell powershell -NoProfile -Command "cat mbackup.fsproj")
        this works.
      - 此时不应有 <
        try do everything in powershell.
        makefile in windows sucks.
      - upgrade and --version both works. tested in win 10 VM.
      - set HOME dir still not working.
        Could not create directory '/home/IEUser/.ssh'.
        c# - Set environment variables for a process - Stack Overflow
        https://stackoverflow.com/questions/14553830/set-environment-variables-for-a-process
        System.Environment.SetEnvironmentVariable changes the environment variables of the current process.
        If you want to change the variables of a process you create, just use the EnvironmentVariables dictionary property.
        still fail.
        try set it to unix style path.
        neither windows path nor unix path works.

        windows - Could not create directory '/home/username/.ssh' - Stack Overflow
        https://stackoverflow.com/questions/36011084/could-not-create-directory-home-username-ssh
        try create etc/nsswitch.conf in rsync distribution.
        content:
        db_home: env windows cygwin desc
        this doesn't work.

        To test this problem, just run this command and see the default file name:
         & 'C:\Program Files\mbackup\rsync-w64\usr\bin\ssh-keygen.exe'

        windows - vagrant up "Error: Could not create directory '/home/username/.ssh'" when using cygwin - Stack Overflow
        https://stackoverflow.com/questions/34754717/vagrant-up-error-could-not-create-directory-home-username-ssh-when-using

        Under Windows, RSync will try and update the %HOME%/.ssh/known_hosts file. If %HOME% is not defined as one of your
        environment variables, it may try and add/update that file where it has no permissions, and fail. Solution: Set user
        environment variable HOME to be %USERPROFILE%.

        // set HOME dir on system level works on B75I3. maybe when rsync invoke ssh, env variable is lost.
        // if system level env variable is required, set HOME when msi is installed if HOME is not set.

        in admin powershell,
        setx HOME ^%USERPROFILE^%
        // try mbackup to remote node, nope.
        setx HOME $env:userprofile
        // try mbackup to remote node, nope.

        how to make HOME work in the Win 10 GUEST?
        TODO Maybe I need to create the rsync zip file using a Dockerfile. LATER.
        And include bash or getenv tool for debugging. Does my rsync tool include bash? yes.

        on B75I3,
        .\bash.exe
        echo $HOME
        /cygdrive/c/Users/sylecn

        on Win10 Guest,
        .\bash.exe
        bash.exe: warning: could not find /tmp, please create!
        echo $HOME
        /cygdrive/c/Users/IEUser
        // looks HOME var is alright. ssh is not using HOME env variable.
        ssh-keygen works fine in bash shell. so bash setup some variable for use with ssh tools.
        after reboot, run ssh-keygen directly in cmd.exe also works.
        When run ssh.exe from cmd.exe, it still complains about HOME dir.
        Could not create directory '/home/IEUser/.ssh'.

        I think I know the problem. When start bash.exe, it will convert all path in PATH, HOME to cygdrive style.
        When start ssh.exe directly, HOME dir is still in windows style.

        Try set HOME to cygdrive style and run ssh.exe.
        how to set env var for current shell only in windows?
        set HOME=/cygdrive/c/Users/IEUser
        ssh.exe ...
        Could not create directory '/home/IEUser/.ssh'.

        search: mingw HOME
        HOWTO Set the HOME variable for cmd exe | MinGW
        http://www.mingw.org/wiki/HOWTO_Set_the_HOME_variable_for_cmd_exe
        Ubuntu Manpage: ssh — OpenSSH SSH client (remote login program)
        https://manpages.ubuntu.com/manpages/bionic/en/man1/ssh.1.html
        ssh will set HOME env var, not read it.
        Check how ssh get home dir. maybe from /etc/passwd and similar syscall.
        check ssh-keygen source code.
        https://github.com/openssh/openssh-portable/blob/master/ssh-keygen.c
        pw = getpwuid(getuid());
        https://manpages.ubuntu.com/manpages/bionic/en/man3/getpwuid.3.html
        indeed it read config for passwd entry.
        how to bundle passwd file in mingw application?

        Configure sshd on MSYS2 and run it as a Windows service · GitHub
        https://gist.github.com/samhocevar/00eec26d9e9988d080ac
        /etc/nsswitch.conf
        db_home: windows
        // what does this do? I think it only change HOME, doesn't work for getpwuid call.
        http://manpages.ubuntu.com/manpages/precise/man5/nsswitch.conf.5.html
        nsswitch can manage how system read passwd db. cool.

        after updating /etc/nsswitch.conf, all MSYS applications and terminals must be
        closed, as the configuration is only read once at MSYS DLL load time.
        try db_home: windows again. reboot and test.
        it doesn't work.
        why it works on B75I3? what's different?

        this is all about UserKnownHostsFile? just specify a path for this option.
        after adding UserKnownHostsFile to point to existing file, ssh still wants to create /home/<user>/.ssh dir.

        I will leave this problem. It's harmless for my use case.

      - System.ArgumentException: Value does not fall within the expected range.
        this happens when I run dotnet run on B75I3 dev node.
        what is this error?

        this line caused this error:
        processStartInfo.EnvironmentVariables.Add("HOME", userHome)
        you can't add a var if it already exists.

  - mbackup.msi works on B75I3 host.
  - try mbackup.msi on win 10 VM.
    how to require dotnet core 3.0 in .wxs file?
    search: wix NetFxExtension symbol for .net core 3
    asp.net mvc - Wix package ID for .NET core runtime 1.0.3 - Stack Overflow
    https://stackoverflow.com/questions/42908913/wix-package-id-for-net-core-runtime-1-0-3
    requires dotnet core 3 runtime on target node.
    backup to local dir works.
    backup to remote ssh node...works.
    - problems
      - 2019-11-14T10:50:13 ERROR Read/write file failed: System.Private.CoreLib Could not find a part of the path 'C:\Users\IEUser\AppData\Local\mbackup\mbackup.list'.
        Does the dir exist? no. Create it at runtime.
        fixed.
      - backup to remote ssh node.
        Warning: Identity file /cygdrive/c/Users/IEUser/.ssh/id_rsa not accessible: No such file or directory.
        Can't open user config file /cygdrive/c/Users/IEUser/.ssh/config: No such file or directory
        DONE do not specify config file if it doesn't exist.
        DONE fail fast if ssh key doesn't exist. mbackup will only support key based auth.
        DONE ssh options should only be added when backup to remote node.

        it works.
        PS C:\Users\IEUser> &"C:\Program Files\mbackup\publish\mbackup.exe" --target root@sylecn01.emacsos.com:/data/backup/PC-backup/IEUser --ssh-key E:\id_rsa --remote-user sylecn
      - TODO failwith should not be used in UI code. It throws unhandled exception. System.Exception.
        search: f# exit early without shifting to the right
        in haskell, I use maybeT and eitherT etc.

        in f#, you should use exception.

        Environment.Exit(exitCode) also works.

- problems
  - each file require it's own <Component> tag.
    rsync mingw have many files. I seems I need to generate an installer for
    rsync for windows.
  - search: wix only install file if it does not exist
    windows installer - Copy if not exist in WiX - Stack Overflow
    https://stackoverflow.com/questions/1912037/copy-if-not-exist-in-wix
    installer - What is the wix 'KeyPath' attribute? - Stack Overflow
    https://stackoverflow.com/questions/2003043/what-is-the-wix-keypath-attribute/2003366#2003366
    Component Table - Win32 apps | Microsoft Docs
    https://docs.microsoft.com/en-us/windows/win32/msi/component-table?redirectedfrom=MSDN
    How To: Check for .NET Framework Versions
    https://wixtoolset.org/documentation/manual/v3/howtos/redistributables_and_install_checks/check_for_dotnet.html
  - WONTFIX how to unpack zip file to target dir?
    I don't want to create a Component for every file there.

    windows installer - Wix custom action to unzip a file - Stack Overflow
    https://stackoverflow.com/questions/47691477/wix-custom-action-to-unzip-a-file

    the suggestion is to include the files in msi.
    or unzip the files at app runtime.

    I will try create a msi file for mingw64 rsync.
    then use wix bundle to install it as dependency.

    - search: wix install all files in a folder

    - Include all Files in Bin folder in Wix installer - Stack Overflow
      https://stackoverflow.com/questions/36756311/include-all-files-in-bin-folder-in-wix-installer

      create rsync-mingw64-files.wxs
      &"C:\Program Files (x86)\WiX Toolset v3.11\bin\heat.exe" dir "D:\downloads\apps\rsync-w64" -cg RsyncHeatGenerated -dr ProgramFiles64Folder -var var.RsyncSourceDir -gg -nologo -out rsync-mingw64-files.wxs -sw5150

      compile and build rsync-mingw64.msi
      &"C:\Program Files (x86)\WiX Toolset v3.11\bin\candle.exe" -dRsyncSourceDir=D:\downloads\apps\rsync-w64\  .\rsync-mingw64.wxs .\rsync-mingw64-files.wxs
      &"C:\Program Files (x86)\WiX Toolset v3.11\bin\light.exe" .\rsync-mingw64.wixobj .\rsync-mingw64-files.wixobj -o rsync-mingw64.msi

      build is getting complex. I need make and Makefile.
    - problems
      - how to pass in preprocessor variable?
        WiX undefined preprocessor variable - Stack Overflow
        https://stackoverflow.com/questions/2063630/wix-undefined-preprocessor-variable
        To pass values for preprocessor variables like $(var.MyProject.TargetDir), use options like -dMyProject.TargetDir=c:\foo.
      - why rsync-mingw64 is installed to C:\Program Files (x86)\rsync-w64? I expect C:\Program Files\rsync-w64\
        wix-users - 64 Bit program files folder
        http://windows-installer-xml-wix-toolset.687559.n2.nabble.com/64-Bit-program-files-folder-td5948080.html
      - INVALID make pattern rule doesn't work in windows
        it works. it is because my source file is missing.
      - why mbackup.exe is 32 bit component?
        try this:
        dotnet publish -c Release -r win10-x64
        still same error.
        D:\sylecn_docs\projects\mbackup-for-windows\mbackup.wxs(16) : error LGHT0204 : ICE80: This 32BitComponent mbackup.exe uses 64BitDirectory APPLICATIONROOTDIRECTORY
        search: dotnet core how to build x64 exe file
        dotnet is building x64 exe file. it's WIX issue.
        need to add Win64="yes" in <Component> attribute.

  - DONE how to run wix to create msi file?
    two command line tool
    light
    candle
    it's not added in PATH.
    C:\Program Files (x86)\WiX Toolset v3.11\bin\light.exe
    C:\Program Files (x86)\WiX Toolset v3.11\bin\candle.exe

    There is offline doc.
    C:\Program Files (x86)\WiX Toolset v3.11\doc\WiX.chm
    C:\Program Files (x86)\WiX Toolset v3.11\doc\msi.chm

    do read the WiX Tutorial.
    https://www.firegiant.com/wix/tutorial/
    https://www.firegiant.com/wix/tutorial/getting-started/

    &"C:\Program Files (x86)\WiX Toolset v3.11\bin\candle.exe" .\mbackup.wxs
    &"C:\Program Files (x86)\WiX Toolset v3.11\bin\light.exe" .\mbackup.wixobj
    it works.
    mbackup.msi file is created.
  - the mbackup.exe can't run without all those dlls.
    dotnet publish -c Release -r win10-x64 --nologo
    maybe I should build a exe that requires dotnet core 3 to run.
    I don't want to create a big msi.
    try this:
    dotnet publish -c Release --nologo
    and update wix xml files.

    need to add all files in bin\Release\netcoreapp3.0\publish\ dir.
    use heat to create another wxs file.
  - heat can't add some dll.
    "C:\Program Files (x86)\WiX Toolset v3.11\bin\heat.exe" dir bin\Release\netcoreapp3.0\publish -cg MbackupHeatGenerated -dr MBACKUP_PROGRAM_FILES -var var.MbackupPublishDir -gg -nologo -out mbackup-files.wxs -sw5150
    heat.exe : warning HEAT5151 : Could not harvest data from a file that was expected to be an assembly: D:\sylecn_docs\projects\mbackup-for-windows\bin\Release\netcoreapp3.0\publish\Argu.dll. If this file is not an assembly you can ignore this warning. Otherwise, this error detail may be helpful to diagnose the failure: 未能加载文件或程序集“FSharp.Core, Version=4.4.3.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a”或它的某一个依赖项。系统找不到
    指定的文件。
    heat.exe : warning HEAT5151 : Could not harvest data from a file that was expected to be an assembly: D:\sylecn_docs\projects\mbackup-for-windows\bin\Release\netcoreapp3.0\publish\mbackup.dll. If this file is not an assembly 
    you can ignore this warning. Otherwise, this error detail may be helpful to diagnose the failure: 未能加载文件或程序集“System.Runtime, Version=4.2.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a”或它的某一个依赖项。系统
    找不到指定的文件。

    .net - Wix Toolkit: Heat balking about DLL's - Stack Overflow
    https://stackoverflow.com/questions/43992433/wix-toolkit-heat-balking-about-dlls
    com - Cannot register DLL using WiX - Stack Overflow
    https://stackoverflow.com/questions/12463256/cannot-register-dll-using-wix
  - TODO dup file?
    <Component Id="cmp981FC09307D7CB82695815FAAFCC646D" Directory="dir02A4AE56AE905AEEE7E2D4F4410A9748" Guid="{DB94490C-32CA-4B14-A96D-A95D4B557D01}">
                <File Id="fil432C46247A06CCDB891611045DD37081" KeyPath="yes" Source="$(var.MbackupPublishDir)\runtimes\win\lib\netstandard2.0\System.Security.Cryptography.ProtectedData.dll" />
    <Component Id="cmp5190411B4FE56A0E0CE782BE6C1B537E" Directory="dir1B0C588BD61288F160D459591DA01603" Guid="{F0613E24-B22B-4A58-A054-380BDF2B857C}">
                <File Id="filE0C94795A0B5F482BD659FBB774641C7" KeyPath="yes" Source="$(var.MbackupPublishDir)\System.Security.Cryptography.ProtectedData.dll" />
    why there are duplicated dll in publish dir? check publish doc.
    this is the only duplicated file. I will ignore it for now.
  - DONE ssh.exe path escape issue.
    &"C:\Program Files\mbackup\rsync-w64\usr\bin\rsync.exe" -h --stats -togr --delete --delete-excluded --ignore-missing-args --files-from=/cygdrive/c/Users/sylecn/AppData/Local/mbackup/mbackup.list --exclude-from=/cygdrive/c/ProgramData/mbackup/mbackup-default.exclude --log-file=/cygdrive/c/Users/sylecn/AppData/Local/mbackup/mbackup.log -e "/cygdrive/c/Program Files/mbackup/rsync-w64/usr/bin/ssh.exe -F /cygdrive/c/Users/sylecn/.ssh/config -i /cygdrive/c/Users/sylecn/.ssh/id_rsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" --remote-option=--log-file=/var/log/mbackup/B75I3.log --chown=sylecn:sylecn / root@sylecn01.emacsos.com:/data/backup/PC-backup/B75I3/
    rsync: Failed to exec /cygdrive/c/Program: No such file or directory (2)

    &"C:\Program Files\mbackup\rsync-w64\usr\bin\rsync.exe" -h --stats -togr --delete --delete-excluded --ignore-missing-args --files-from=/cygdrive/c/Users/sylecn/AppData/Local/mbackup/mbackup.list --exclude-from=/cygdrive/c/ProgramData/mbackup/mbackup-default.exclude --log-file=/cygdrive/c/Users/sylecn/AppData/Local/mbackup/mbackup.log -e "'/cygdrive/c/Program Files/mbackup/rsync-w64/usr/bin/ssh.exe' -F /cygdrive/c/Users/sylecn/.ssh/config -i /cygdrive/c/Users/sylecn/.ssh/id_rsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" --remote-option=--log-file=/var/log/mbackup/B75I3.log --chown=sylecn:sylecn / root@sylecn01.emacsos.com:/data/backup/PC-backup/B75I3/

** 2019-11-16 support reading remote-user=xxx option from config file.
cli arg > env var > config file.
same with target option.
** 2019-11-13 next todos; scheduled task command line.
- DONE fix TODOs in F# code
- DONE add rsync command and arguments for running in windows.
  i.e. -e etc.

  <Arguments>--stats -togr --chown=sylecn:sylecn --exclude-from=/cygdrive/d/sylecn_docs/texts/configs/rsync-exclude
   --files-from=/cygdrive/d/sylecn_docs/texts/configs/rsync-file-list
    --log-file=/cygdrive/d/sylecn_docs/rsync-b75i3.log
     -e ".\ssh.exe -F c:/users/sylecn/.ssh/config -i c:/Users/sylecn/.ssh/id_rsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null"
      / sheni:/data/backup/server-backup/b75i3/</Arguments>

- DONE test param parsing is working.
  command line param > env var > mbackup default value.
  support --node-name param.
- DONE test run in console and scheduled task.
  run in console works.
  try run in scheduled task.

  SCHTASKS /Create /?
  # run mbackup 15m after user logon.
  SCHTASKS /Create /U <username> /SC ONLOGON /TN mbackup-logon /TR "\"<path\to\mbackup.exe>\" \"args\"" /DELAY 15:00
  # run mbackup at 10am and 4pm.
  SCHTASKS /Create /U <username> /SC DAILY /TN mbackup-morning /TR "\"<path\to\mbackup.exe>\" \"args\"" /ST 10:00 /ET 13:00 /K
  SCHTASKS /Create /U <username> /SC DAILY /TN mbackup-afternoon /TR "\"<path\to\mbackup.exe>\" \"args\"" /ST 16:00 /ET 19:00 /K

  # debug purpose, one time only
  SCHTASKS /Create /SC ONCE /TN mbackup-test /TR "\"<path\to\mbackup.exe>\" \"args\"" /ST 19:18

  in powershell,
  $mbackupexe = "D:\sylecn_docs\projects\mbackup-for-windows\bin\Debug\netcoreapp3.0\mbackup-for-windows.exe"
  SCHTASKS /Create /SC ONCE /TN mbackup-test /TR $mbackupexe /ST 19:21
  default run as user is current user.

  in admin powershell,
  SCHTASKS /Create /SC ONCE /TN mbackup-test /TR $mbackupexe /ST 19:32 /NP
  /NP option requires admin privilledge.
  it works.

  to delete it,
  schtasks /delete /tn mbackup-test

  - problems
    - how many scheduled task to run on a multi-user PC?
      each user have its own user-default.list expansion.
      should I iter over all users on PC?
      I think only current user can get it's profile dir and special dirs.
    - maybe config mbackup to run after user logon. with 15minute delay.
      always run as current user.
    - how to not require any param when running mbackup.exe?
      put TARGET and other option in a config file?
      define system level TARGET env variable.

      use config file is easier for user to edit and making the change effective.
      userConfigDir / mbackup.conf

      search: F# read config file
      FSharp.Configuration
      http://fsprojects.github.io/FSharp.Configuration/
    - FSharp.Configuration missing reference to System.Runtime.Caching
      search: how to reference System.Runtime.Caching for dotnet core project
      https://www.nuget.org/packages/System.Runtime.Caching/
      dotnet add package System.Runtime.Caching --version 4.6.0
      should I add 4.0.0? Can I use a higher version?

      still not compatible.
      search: use FSharp.Configuration with dotnet core 3
    - give up on FSharp.Configuration.
    - try this:
      FsConfig
      https://www.demystifyfp.com/FsConfig/
      AppSettings is only supported in V0.0.6 or below.
      try F# AppSettings directly.
      If nothing is easy to use, write my own parser.
      Support similar config format as python wells lib.
    - ConfigurationManager.AppSettings Property (System.Configuration) | Microsoft Docs
      https://docs.microsoft.com/en-us/dotnet/api/system.configuration.configurationmanager.appsettings?view=netframework-4.8
      where should I save the App.config xml file?
      search: what is app.config
      App.Config: Basics and Best Practices - SubMain Blog
      https://blog.submain.com/app-config-basics-best-practices/
      What is App.config in C#.NET? How to use it? - Stack Overflow
      https://stackoverflow.com/questions/13043530/what-is-app-config-in-c-net-how-to-use-it
      Okay. This is not what I want.
      This is for application configuration (rarely change), not for user configuration (can change any time).
      dotnet will create <yourapp>.exe.config from your App.config file.
    - how to use multiple files in F# dotnet core project?
      search: f# module and namespace
      search: f# module
      Modules - F# | Microsoft Docs
      https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/modules
    - SCHTASKS /Create /SC ONCE /TN mbackup-test /TR $mbackupexe /ST 19:21
      this runs, it will show a console window. how to not show console window?

      search: schedule tasks do not show console window

      In the “General” tab, under the “Security options” section, select the Run
      whether user is logged on or not option. (This is the option that will make
      the command window not to appear when the task runs automatically.)

      try /NP option.

- create an installer. The installer should add scheduled task on install and
  delete scheduled task on removal.

- problems
  - how to write unit test in F#?
    Unit testing F# in .NET Core with dotnet test and NUnit - .NET Core | Microsoft Docs
    https://docs.microsoft.com/en-us/dotnet/core/testing/unit-testing-fsharp-with-nunit
    mkdir mbackup-tests
    cd mbackup-tests
    dotnet new nunit -lang F#
    //wait.
    dotnet test

    dotnet add reference ../mbackup-for-windows.fsproj
    dotnet build  # this will build both ref project and test project.
    dotnet test
  - how to support short option names?
    --dry-run -n
  - 
** 2019-11-14 support mbackup.txt config.
config file will have wells config file format.
empty lines and comment lines are ignored.
space around = is ignored.
if there are quotes on value side, it's trimmed.

target=user@host:port/path/
# target = user@host:port/path/
abc.def = some value
abc.def.ghi = 1233.45
foo = yes

env variable will be
TARGET
ABC_DEF
ABC_DEF_GHI
FOO

in F# code,
conf = WellsConfig("C:\path\to\file.txt")
conf.GetStr("target")
conf.GetFloat("abc.def.ghi")
conf.GetBool("foo")

it works.

** 2019-11-13 extra user default list.
Documents is replaced by real path.
Downloads
Pictures

User should always use forward slash in backup list and exclude list files.
These path are supported:
C:\Foo\Bar
C:/Foo/Bar
Pictures/Saved Pictures
Documents/

** 2019-11-13 build mbackup.list file from file list and exclude lists.
* wontfix                                                             :entry: