Newer
Older
** 2019-11-13 how to run it in dev env?
dotnet run -- --dry-run --itemize-changes --target d:\backup
dotnet run -- -n -i --target d:\backup
try an ssh run:
dotnet run -- -n --target root@sylecn01.emacsos.com:/data/backup/PC-backup/B75I3/
it works.
file list works.
ssh transfer works.
remote logging works.
read target from config file works. now I can just run
dotnet run -- -i
- rsync
https://www.samba.org/ftp/rsync/rsync.html
- Basic Editing in Visual Studio Code
https://code.visualstudio.com/docs/editor/codebasics
Get Started with F# in Visual Studio Code | Microsoft Docs
https://docs.microsoft.com/en-us/dotnet/fsharp/get-started/get-started-vscode
install Ionide-fsharp extension.
- Argu
http://fsprojects.github.io/Argu/
Tutorial
http://fsprojects.github.io/Argu/tutorial.html
--help doesn't work, need error handler.
Argu/Program.fs at master · fsprojects/Argu · GitHub
https://github.com/fsprojects/Argu/blob/master/samples/Argu.Samples.LS/Program.fs
- F#
F# Language Reference - F# | Microsoft Docs
https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/
F# Collection Types - F# | Microsoft Docs
https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/fsharp-collection-types
Modules - F# | Microsoft Docs
https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/modules
https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/namespaces
fsharp-cheatsheet
https://dungpa.github.io/fsharp-cheatsheet/
Literals - F# | Microsoft Docs
https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/literals
[|' '|] is an array of character, length is 1.
F# list is not the same as array. You can convert array to list via Array.toList
String Class (System) | Microsoft Docs
https://docs.microsoft.com/en-us/dotnet/api/system.string?view=netframework-4.8
Environment.SpecialFolder
https://docs.microsoft.com/en-us/dotnet/api/system.environment.specialfolder?view=netframework-4.8
ApplicationData
LocalApplicationData
MyDocuments
MyPictures
DesktopDirectory
Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData)
DateTime.ToString Method (System) | Microsoft Docs
https://docs.microsoft.com/en-us/dotnet/api/system.datetime.tostring?view=netframework-4.8
Standard Date and Time Format Strings | Microsoft Docs
https://docs.microsoft.com/en-us/dotnet/standard/base-types/standard-date-and-time-format-strings?view=netframework-4.8
The Option type | F# for fun and profit
https://fsharpforfunandprofit.com/posts/the-option-type/
Pattern Matching - F# | Microsoft Docs
https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/pattern-matching
- FSharpLint
http://fsprojects.github.io/FSharpLint/
http://fsprojects.github.io/FSharpLint/index.html
- FSharp.Configuration
http://fsprojects.github.io/FSharp.Configuration/
dotnet add package FSharp.Configuration --version 1.5.0
http://fsprojects.github.io/FSharp.Configuration/IniTypeProvider.html
I will use the ini provider.
user don't need to escape string or pay attension to spaces in ini file.
this lib doesn't work with dotnet core 3.
- Formatted text using printf | F# for fun and profit
https://fsharpforfunandprofit.com/posts/printf/
- Wix: a tool to create msi file
WiX Toolset
https://wixtoolset.org/
Windows Installer XML (WiX)
https://wixtoolset.org/documentation/manual/v3/main/
To learn the basics, do read the WiX Tutorial.
https://www.firegiant.com/wix/tutorial/
https://www.firegiant.com/wix/tutorial/getting-started/
Then you can follow the how-tos.
How To: Implement a Major Upgrade In Your Installer
https://wixtoolset.org/documentation/manual/v3/howtos/updates/major_upgrade.html
** 2019-11-13 install dir layout.
C:\Program Files\mbackup\rsync-w64\usr\bin\rsync.exe
C:\Program Files\mbackup\rsync-w64\usr\bin\ssh.exe
C:\Program Files\mbackup\publish\mbackup.exe
C:\Program Files\mbackup\publish\mbackup.dll
C:\ProgramData\mbackup\mbackup-default.exclude
C:\ProgramData\mbackup\mbackup-default.list
C:\ProgramData\mbackup\user-default.list
C:\ProgramData\mbackup\mbackup.txt
- optional user configs
like /etc/mbackup/ in linux
C:\ProgramData\mbackup\local.list
C:\ProgramData\mbackup\local.exclude
like ~/.mbackup/ in linux
<My Documents>/mbackup/local.list
<My Documents>/mbackup/local.exclude
** 2019-11-14 notes :development:
- Argu optional param support.
if option Target is optional, when try to get its value, you should use
results.TryGetResult Target
or
results.GetResult(Target, defaultValue = xxx)
- WiX wxs file
<Package InstallerVersion="300" Compressed="yes" Platform="x64"/>
Platform="x64" has the same meaning as candle -arch x64.
- Wix for creating installer for 64bit app, you need to add in <Component> attribute
Win64="yes"
use candle -arch x64 can add Win64="yes" automatically if there is no Win64 attribute specified in Component.
wix3.6 - WiX Heat tool, create Component with Win64 attribute - Stack Overflow
https://stackoverflow.com/questions/11981498/wix-heat-tool-create-component-with-win64-attribute
- Wix to install file in another name, add Name attribute in <File>.
<File Id="mbackup.exe" Name="mbackup.exe" Source="bin\Release\netcoreapp3.0\win10-x64\mbackup-for-windows.exe" KeyPath="yes" Checksum="yes"/>
- vscode powershell integrated console is not the same as powershell.
it doesn't read $profile, and can't run ". $profile" command.
it's best to press + to create a regular powershell terminal.
* later :entry:
** 2019-11-14 supports expand Downloads dir in user-default.list
** 2019-11-14 vscode f# doesn't support open a module
it can only support open a namespace.
using the vscode Ionide-fsharp extension.
** 2019-11-16 ssh.exe Could not create directory '/home/IEUser/.ssh' error.
when rsync.exe or ssh.exe is run from cmd.exe, it will always show this error.
HOME env var is defined properly.
- create /etc/nsswitch.conf with the following content didn't fix it.
db_home: windows
- openssh get user's home dir from getpwuid call.
Check how ssh get home dir. maybe from /etc/passwd and similar syscall.
check ssh-keygen source code.
https://github.com/openssh/openssh-portable/blob/master/ssh-keygen.c
pw = getpwuid(getuid());
https://manpages.ubuntu.com/manpages/bionic/en/man3/getpwuid.3.html
indeed it read config for passwd entry.
how to bundle passwd file in mingw application?
** 2019-11-15 f# indent is difficult in vscode.
vscode should at least always indent using space for F#.
- try a code format tool.
GitHub - fsprojects/fantomas: FSharp source code formatter
https://github.com/fsprojects/fantomas
dotnet tool install fantomas-tool -g
- there is context menu > format document.
lonide-fsharp already support it.
format document is on whole buffer. not just selected region.
some code is not formatted properly.
https://code.visualstudio.com/docs/editor/codebasics
Formatting.
Format Document (Ctrl+Shift+I) - Format the entire active file.
Format Selection (Ctrl+K Ctrl+F) - Format the selected text.
but there is not format selection in f# buffer.
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
** TODO 2019-11-15 additionally support <My Documents>/mbackup/local.list file.
This file is easier to open and backup.
no. programdata should be good enough.
but this is per user extra config file.
each user could have their own local file to backup.
one user don't want to manage all other user's local backup list.
yes. I should support this.
Both local.list and local.exclude.
** 2019-11-14 learn how to create an msi installer.
- WiX Toolset
https://wixtoolset.org/
c# - How to create MSI installer for ASP.NET Core application using Wix Toolset - Stack Overflow
https://stackoverflow.com/questions/46878752/how-to-create-msi-installer-for-asp-net-core-application-using-wix-toolset
Windows Installer XML (WiX)
https://wixtoolset.org/documentation/manual/v3/main/
- Introducing dotGet: an installer for .NET Core CLI apps
https://medium.com/@tonerdo/introducing-dotget-an-installer-for-net-core-cli-apps-9ba79ac7cd89
requires nuget. then it can install your dotnet core console project.
this is not what I need here.
- Packaging a self-contained .NET Core app for Windows Installer
https://nblumhardt.com/2017/04/netcore-msi/
This is the first time I’ve tried packaging a .NET Core app for the Windows
Installer, so I thought I’d write these brief notes to help anyone else
interested in taking the same path.
- should I include rsync-mingw64 in the same msi file?
distributing it alone is not very useful, since people can just use msys2 or cygwin.
If I include it in the same msi, I don't need to use wix bundle, and I can install rsync to the same dir as mbackup.
it seems cleaner this way. Just like a VPN client bundle openvpn or a python program bundle python.
yes. include it in the same msi and install in a sub dir.
- try Wix.
- install wix.
requires .net 3.5 windows feature.
- Building Installation Package Bundles
https://wixtoolset.org/documentation/manual/v3/bundle/
- How To Guides
https://wixtoolset.org/documentation/manual/v3/howtos/
- How To: Add a File to Your Installer
https://wixtoolset.org/documentation/manual/v3/howtos/files_and_registry/add_a_file.html
build mbackup binaries
dotnet publish -c Release
- TODO how to show a message when installer is finished succesfully?
currently it installs very fast and just exit without any user notification.
maybe show a installer window and let user click Next and Finish.
- TODO how to add some dir to PATH?
I only have one exe. Maybe just create a start menu or desktop shortcut?
DONE create desktop shortcut to mbackup.exe
installer - Create shortcut to desktop using WiX - Stack Overflow
https://stackoverflow.com/questions/11868499/create-shortcut-to-desktop-using-wix
it works.
install will install two shortcuts.
uninstall will remove both shortcuts.
- DONE how to support upgrade when I click a new msi?
currently every msi is a standalone msi and will always install fresh.
is the UpgradeCode used for this?
does <Product> version have any effect on generated msi file?
how to integrate this version with dotnet application version?
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
search: wix msi how to support upgrade
How To: Implement a Major Upgrade In Your Installer
https://wixtoolset.org/documentation/manual/v3/howtos/updates/major_upgrade.html
- how to support L10N for wxs file? DowngradeErrorMessage
<MajorUpgrade AllowDowngrades="no" DowngradeErrorMessage="!(loc.NewerVersionInstalled)"
AllowSameVersionUpgrades="no" />
- in my application, downgrade should be supported.
- note: <MajorUpgrade AllowDowngrades="yes"/> element should come after <Media>.
- add --version support.
search: dotnet application where to put version info and support --version parameter
https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-build
Build the project and set version 1.2.3.4 as a build parameter using the -p MSBuild option:
dotnet build -p:Version=1.2.3.4
how to get the Version in code?
Get App Version in .NET Core - Edi.Wang
https://edi.wang/post/2018/9/27/get-app-version-net-core
First, let's see a typical .NET Core project file with versioning. We can define
the version numbers for AssemblyVersion, FileVersion and Version in the project
file (.csproj).
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp2.1</TargetFramework>
<AssemblyVersion>1.1.1.1</AssemblyVersion>
<FileVersion>2.2.2.2</FileVersion>
<Version>3.3.3.3-xyz</Version>
</PropertyGroup>
</Project>
- problems
- Could not create directory '/home/IEUser/.ssh'.
how to make this error gone?
check ssh option. maybe just set HOME env variable if it doesn't exist when starting rsync process?
search: dotnet set environment variable
Environment.SetEnvironmentVariable Method
- how to get Version for use in Makefile?
$(file < mbackup.fsproj) doesn't get expanded?
try use $(shell) instead.
FSPROJ := $(shell powershell -NoProfile -Command {cat mbackup.fsproj})
still not working.
FSPROJ has value of cat mbackup.fsproj
search: makefile read file in windows
no result.
Make is not well supported in windows.
FSPROJ := $(shell powershell -NoProfile -Command "cat mbackup.fsproj")
this works.
- 此时不应有 <
try do everything in powershell.
makefile in windows sucks.
- upgrade and --version both works. tested in win 10 VM.
- set HOME dir still not working.
Could not create directory '/home/IEUser/.ssh'.
c# - Set environment variables for a process - Stack Overflow
https://stackoverflow.com/questions/14553830/set-environment-variables-for-a-process
System.Environment.SetEnvironmentVariable changes the environment variables of the current process.
If you want to change the variables of a process you create, just use the EnvironmentVariables dictionary property.
still fail.
try set it to unix style path.
neither windows path nor unix path works.
windows - Could not create directory '/home/username/.ssh' - Stack Overflow
https://stackoverflow.com/questions/36011084/could-not-create-directory-home-username-ssh
try create etc/nsswitch.conf in rsync distribution.
content:
db_home: env windows cygwin desc
this doesn't work.
To test this problem, just run this command and see the default file name:
& 'C:\Program Files\mbackup\rsync-w64\usr\bin\ssh-keygen.exe'
windows - vagrant up "Error: Could not create directory '/home/username/.ssh'" when using cygwin - Stack Overflow
https://stackoverflow.com/questions/34754717/vagrant-up-error-could-not-create-directory-home-username-ssh-when-using
Under Windows, RSync will try and update the %HOME%/.ssh/known_hosts file. If %HOME% is not defined as one of your
environment variables, it may try and add/update that file where it has no permissions, and fail. Solution: Set user
environment variable HOME to be %USERPROFILE%.
// set HOME dir on system level works on B75I3. maybe when rsync invoke ssh, env variable is lost.
// if system level env variable is required, set HOME when msi is installed if HOME is not set.
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
in admin powershell,
setx HOME ^%USERPROFILE^%
// try mbackup to remote node, nope.
setx HOME $env:userprofile
// try mbackup to remote node, nope.
how to make HOME work in the Win 10 GUEST?
TODO Maybe I need to create the rsync zip file using a Dockerfile. LATER.
And include bash or getenv tool for debugging. Does my rsync tool include bash? yes.
on B75I3,
.\bash.exe
echo $HOME
/cygdrive/c/Users/sylecn
on Win10 Guest,
.\bash.exe
bash.exe: warning: could not find /tmp, please create!
echo $HOME
/cygdrive/c/Users/IEUser
// looks HOME var is alright. ssh is not using HOME env variable.
ssh-keygen works fine in bash shell. so bash setup some variable for use with ssh tools.
after reboot, run ssh-keygen directly in cmd.exe also works.
When run ssh.exe from cmd.exe, it still complains about HOME dir.
Could not create directory '/home/IEUser/.ssh'.
I think I know the problem. When start bash.exe, it will convert all path in PATH, HOME to cygdrive style.
When start ssh.exe directly, HOME dir is still in windows style.
Try set HOME to cygdrive style and run ssh.exe.
how to set env var for current shell only in windows?
set HOME=/cygdrive/c/Users/IEUser
ssh.exe ...
Could not create directory '/home/IEUser/.ssh'.
search: mingw HOME
HOWTO Set the HOME variable for cmd exe | MinGW
http://www.mingw.org/wiki/HOWTO_Set_the_HOME_variable_for_cmd_exe
Ubuntu Manpage: ssh — OpenSSH SSH client (remote login program)
https://manpages.ubuntu.com/manpages/bionic/en/man1/ssh.1.html
ssh will set HOME env var, not read it.
Check how ssh get home dir. maybe from /etc/passwd and similar syscall.
check ssh-keygen source code.
https://github.com/openssh/openssh-portable/blob/master/ssh-keygen.c
pw = getpwuid(getuid());
https://manpages.ubuntu.com/manpages/bionic/en/man3/getpwuid.3.html
indeed it read config for passwd entry.
how to bundle passwd file in mingw application?
Configure sshd on MSYS2 and run it as a Windows service · GitHub
https://gist.github.com/samhocevar/00eec26d9e9988d080ac
/etc/nsswitch.conf
db_home: windows
// what does this do? I think it only change HOME, doesn't work for getpwuid call.
http://manpages.ubuntu.com/manpages/precise/man5/nsswitch.conf.5.html
nsswitch can manage how system read passwd db. cool.
after updating /etc/nsswitch.conf, all MSYS applications and terminals must be
closed, as the configuration is only read once at MSYS DLL load time.
try db_home: windows again. reboot and test.
it doesn't work.
why it works on B75I3? what's different?
this is all about UserKnownHostsFile? just specify a path for this option.
after adding UserKnownHostsFile to point to existing file, ssh still wants to create /home/<user>/.ssh dir.
I will leave this problem. It's harmless for my use case.
- System.ArgumentException: Value does not fall within the expected range.
this happens when I run dotnet run on B75I3 dev node.
what is this error?
this line caused this error:
processStartInfo.EnvironmentVariables.Add("HOME", userHome)
you can't add a var if it already exists.
- mbackup.msi works on B75I3 host.
- try mbackup.msi on win 10 VM.
how to require dotnet core 3.0 in .wxs file?
search: wix NetFxExtension symbol for .net core 3
asp.net mvc - Wix package ID for .NET core runtime 1.0.3 - Stack Overflow
https://stackoverflow.com/questions/42908913/wix-package-id-for-net-core-runtime-1-0-3
requires dotnet core 3 runtime on target node.
backup to local dir works.
backup to remote ssh node...works.
- problems
- 2019-11-14T10:50:13 ERROR Read/write file failed: System.Private.CoreLib Could not find a part of the path 'C:\Users\IEUser\AppData\Local\mbackup\mbackup.list'.
Does the dir exist? no. Create it at runtime.
fixed.
- backup to remote ssh node.
Warning: Identity file /cygdrive/c/Users/IEUser/.ssh/id_rsa not accessible: No such file or directory.
Can't open user config file /cygdrive/c/Users/IEUser/.ssh/config: No such file or directory
DONE do not specify config file if it doesn't exist.
DONE fail fast if ssh key doesn't exist. mbackup will only support key based auth.
DONE ssh options should only be added when backup to remote node.
it works.
PS C:\Users\IEUser> &"C:\Program Files\mbackup\publish\mbackup.exe" --target root@sylecn01.emacsos.com:/data/backup/PC-backup/IEUser --ssh-key E:\id_rsa --remote-user sylecn
- TODO failwith should not be used in UI code. It throws unhandled exception. System.Exception.
search: f# exit early without shifting to the right
in haskell, I use maybeT and eitherT etc.
in f#, you should use exception.
Environment.Exit(exitCode) also works.
- problems
- each file require it's own <Component> tag.
rsync mingw have many files. I seems I need to generate an installer for
rsync for windows.
- search: wix only install file if it does not exist
windows installer - Copy if not exist in WiX - Stack Overflow
https://stackoverflow.com/questions/1912037/copy-if-not-exist-in-wix
installer - What is the wix 'KeyPath' attribute? - Stack Overflow
https://stackoverflow.com/questions/2003043/what-is-the-wix-keypath-attribute/2003366#2003366
Component Table - Win32 apps | Microsoft Docs
https://docs.microsoft.com/en-us/windows/win32/msi/component-table?redirectedfrom=MSDN
How To: Check for .NET Framework Versions
https://wixtoolset.org/documentation/manual/v3/howtos/redistributables_and_install_checks/check_for_dotnet.html
- WONTFIX how to unpack zip file to target dir?
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
I don't want to create a Component for every file there.
windows installer - Wix custom action to unzip a file - Stack Overflow
https://stackoverflow.com/questions/47691477/wix-custom-action-to-unzip-a-file
the suggestion is to include the files in msi.
or unzip the files at app runtime.
I will try create a msi file for mingw64 rsync.
then use wix bundle to install it as dependency.
- search: wix install all files in a folder
- Include all Files in Bin folder in Wix installer - Stack Overflow
https://stackoverflow.com/questions/36756311/include-all-files-in-bin-folder-in-wix-installer
create rsync-mingw64-files.wxs
&"C:\Program Files (x86)\WiX Toolset v3.11\bin\heat.exe" dir "D:\downloads\apps\rsync-w64" -cg RsyncHeatGenerated -dr ProgramFiles64Folder -var var.RsyncSourceDir -gg -nologo -out rsync-mingw64-files.wxs -sw5150
compile and build rsync-mingw64.msi
&"C:\Program Files (x86)\WiX Toolset v3.11\bin\candle.exe" -dRsyncSourceDir=D:\downloads\apps\rsync-w64\ .\rsync-mingw64.wxs .\rsync-mingw64-files.wxs
&"C:\Program Files (x86)\WiX Toolset v3.11\bin\light.exe" .\rsync-mingw64.wixobj .\rsync-mingw64-files.wixobj -o rsync-mingw64.msi
build is getting complex. I need make and Makefile.
- problems
- how to pass in preprocessor variable?
WiX undefined preprocessor variable - Stack Overflow
https://stackoverflow.com/questions/2063630/wix-undefined-preprocessor-variable
To pass values for preprocessor variables like $(var.MyProject.TargetDir), use options like -dMyProject.TargetDir=c:\foo.
- why rsync-mingw64 is installed to C:\Program Files (x86)\rsync-w64? I expect C:\Program Files\rsync-w64\
wix-users - 64 Bit program files folder
http://windows-installer-xml-wix-toolset.687559.n2.nabble.com/64-Bit-program-files-folder-td5948080.html
- INVALID make pattern rule doesn't work in windows
it works. it is because my source file is missing.
- why mbackup.exe is 32 bit component?
try this:
dotnet publish -c Release -r win10-x64
still same error.
D:\sylecn_docs\projects\mbackup-for-windows\mbackup.wxs(16) : error LGHT0204 : ICE80: This 32BitComponent mbackup.exe uses 64BitDirectory APPLICATIONROOTDIRECTORY
search: dotnet core how to build x64 exe file
dotnet is building x64 exe file. it's WIX issue.
need to add Win64="yes" in <Component> attribute.
- DONE how to run wix to create msi file?
two command line tool
light
candle
it's not added in PATH.
C:\Program Files (x86)\WiX Toolset v3.11\bin\light.exe
C:\Program Files (x86)\WiX Toolset v3.11\bin\candle.exe
There is offline doc.
C:\Program Files (x86)\WiX Toolset v3.11\doc\WiX.chm
C:\Program Files (x86)\WiX Toolset v3.11\doc\msi.chm
do read the WiX Tutorial.
https://www.firegiant.com/wix/tutorial/
https://www.firegiant.com/wix/tutorial/getting-started/
&"C:\Program Files (x86)\WiX Toolset v3.11\bin\candle.exe" .\mbackup.wxs
&"C:\Program Files (x86)\WiX Toolset v3.11\bin\light.exe" .\mbackup.wixobj
it works.
mbackup.msi file is created.
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
- the mbackup.exe can't run without all those dlls.
dotnet publish -c Release -r win10-x64 --nologo
maybe I should build a exe that requires dotnet core 3 to run.
I don't want to create a big msi.
try this:
dotnet publish -c Release --nologo
and update wix xml files.
need to add all files in bin\Release\netcoreapp3.0\publish\ dir.
use heat to create another wxs file.
- heat can't add some dll.
"C:\Program Files (x86)\WiX Toolset v3.11\bin\heat.exe" dir bin\Release\netcoreapp3.0\publish -cg MbackupHeatGenerated -dr MBACKUP_PROGRAM_FILES -var var.MbackupPublishDir -gg -nologo -out mbackup-files.wxs -sw5150
heat.exe : warning HEAT5151 : Could not harvest data from a file that was expected to be an assembly: D:\sylecn_docs\projects\mbackup-for-windows\bin\Release\netcoreapp3.0\publish\Argu.dll. If this file is not an assembly you can ignore this warning. Otherwise, this error detail may be helpful to diagnose the failure: 未能加载文件或程序集“FSharp.Core, Version=4.4.3.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a”或它的某一个依赖项。系统找不到
指定的文件。
heat.exe : warning HEAT5151 : Could not harvest data from a file that was expected to be an assembly: D:\sylecn_docs\projects\mbackup-for-windows\bin\Release\netcoreapp3.0\publish\mbackup.dll. If this file is not an assembly
you can ignore this warning. Otherwise, this error detail may be helpful to diagnose the failure: 未能加载文件或程序集“System.Runtime, Version=4.2.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a”或它的某一个依赖项。系统
找不到指定的文件。
.net - Wix Toolkit: Heat balking about DLL's - Stack Overflow
https://stackoverflow.com/questions/43992433/wix-toolkit-heat-balking-about-dlls
com - Cannot register DLL using WiX - Stack Overflow
https://stackoverflow.com/questions/12463256/cannot-register-dll-using-wix
- TODO dup file?
<Component Id="cmp981FC09307D7CB82695815FAAFCC646D" Directory="dir02A4AE56AE905AEEE7E2D4F4410A9748" Guid="{DB94490C-32CA-4B14-A96D-A95D4B557D01}">
<File Id="fil432C46247A06CCDB891611045DD37081" KeyPath="yes" Source="$(var.MbackupPublishDir)\runtimes\win\lib\netstandard2.0\System.Security.Cryptography.ProtectedData.dll" />
<Component Id="cmp5190411B4FE56A0E0CE782BE6C1B537E" Directory="dir1B0C588BD61288F160D459591DA01603" Guid="{F0613E24-B22B-4A58-A054-380BDF2B857C}">
<File Id="filE0C94795A0B5F482BD659FBB774641C7" KeyPath="yes" Source="$(var.MbackupPublishDir)\System.Security.Cryptography.ProtectedData.dll" />
why there are duplicated dll in publish dir? check publish doc.
this is the only duplicated file. I will ignore it for now.
- DONE ssh.exe path escape issue.
&"C:\Program Files\mbackup\rsync-w64\usr\bin\rsync.exe" -h --stats -togr --delete --delete-excluded --ignore-missing-args --files-from=/cygdrive/c/Users/sylecn/AppData/Local/mbackup/mbackup.list --exclude-from=/cygdrive/c/ProgramData/mbackup/mbackup-default.exclude --log-file=/cygdrive/c/Users/sylecn/AppData/Local/mbackup/mbackup.log -e "/cygdrive/c/Program Files/mbackup/rsync-w64/usr/bin/ssh.exe -F /cygdrive/c/Users/sylecn/.ssh/config -i /cygdrive/c/Users/sylecn/.ssh/id_rsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" --remote-option=--log-file=/var/log/mbackup/B75I3.log --chown=sylecn:sylecn / root@sylecn01.emacsos.com:/data/backup/PC-backup/B75I3/
rsync: Failed to exec /cygdrive/c/Program: No such file or directory (2)
&"C:\Program Files\mbackup\rsync-w64\usr\bin\rsync.exe" -h --stats -togr --delete --delete-excluded --ignore-missing-args --files-from=/cygdrive/c/Users/sylecn/AppData/Local/mbackup/mbackup.list --exclude-from=/cygdrive/c/ProgramData/mbackup/mbackup-default.exclude --log-file=/cygdrive/c/Users/sylecn/AppData/Local/mbackup/mbackup.log -e "'/cygdrive/c/Program Files/mbackup/rsync-w64/usr/bin/ssh.exe' -F /cygdrive/c/Users/sylecn/.ssh/config -i /cygdrive/c/Users/sylecn/.ssh/id_rsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" --remote-option=--log-file=/var/log/mbackup/B75I3.log --chown=sylecn:sylecn / root@sylecn01.emacsos.com:/data/backup/PC-backup/B75I3/
** 2019-11-12 make code work in a specific dir. then create an installer.
- bundle dotnet core 3 with installer.
- install binary files to %programfiles%
- install mbackup config files to %programdata%
- install scheduled tasks as admin
SCHTASKS /Create /?
# run mbackup 15m after user logon.
SCHTASKS /Create /NP /SC ONLOGON /TN mbackup-logon /TR "\"<path\to\mbackup.exe>\" \"args\"" /DELAY 15:00
SCHTASKS /Create /NP /SC DAILY /TN mbackup-morning /TR "\"<path\to\mbackup.exe>\" \"args\"" /ST 10:00 /ET 13:00 /K
SCHTASKS /Create /NP /SC DAILY /TN mbackup-afternoon /TR "\"<path\to\mbackup.exe>\" \"args\"" /ST 16:00 /ET 19:00 /K
SCHTASKS /Create /NP /SC ONCE /TN mbackup-test /TR "\"<path\to\mbackup.exe>\" \"args\"" /ST 19:18
In admin powershell,
SCHTASKS /Create /NP /SC ONCE /TN mbackup-test /TR $mbackupexe /ST 19:32
/NP option requires admin privilledge.
it works.
to delete it,
schtasks /delete /tn mbackup-test
- config is installed/saved to %appdata% roaming dir.
- problems
- config should be system wide. %appdata% is per-user dir.
where should I save global config file in windows 10?
Windows uses the %APPDATA% directory for user specific application
configuration files. The %PROGRAMDATA% folder is used for system wide
application configuration files, though it appears that Windows Store
apps may not be able to use %PROGRAMDATA%.
C:\ProgramData\mbackup\mbackup-default.list
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
- on B75I3, rsync is here.
D:\downloads\apps\rsync-w64-3.1.3-2-standalone\usr\bin\rsync.exe
scheduled task command is:
<Actions Context="Author">
<Exec>
<Command>D:\downloads\apps\rsync-w64-3.1.3-2-standalone\usr\bin\rsync.exe</Command>
<Arguments>--stats -togr --chown=sylecn:sylecn --exclude-from=/cygdrive/d/sylecn_docs/texts/configs/rsync-exclude --files-from=/cygdrive/d/sylecn_docs/texts/configs/rsync-file-list --log-file=/cygdrive/d/sylecn_docs/rsync-b75i3.log -e ".\ssh.exe -F c:/users/sylecn/.ssh/config -i c:/Users/sylecn/.ssh/id_rsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" / sheni:/data/backup/server-backup/b75i3/</Arguments>
<WorkingDirectory>D:\downloads\apps\rsync-w64-3.1.3-2-standalone\usr\bin\</WorkingDirectory>
</Exec>
</Actions>
- I don't want to bundle python with mbackup for windows, so I will use
C# and dotnet core 3 to create mbackup console executable.
mbackup --cron
# run the command as if run in scheduled task.
mbackup
# run backup command interactively.
create new console project.
dotnet new console -lang=F#
Try F# this time. it's said dotnet have very good F# support.
It's a small program, should be easy to handle.
- implementation
- install-package argu
can't find package.
https://www.nuget.org/packages/Argu
try this:
dotnet add package Argu --version 5.5.0
it works.
- how to change output exe file name?
can't find it. won't fix. default name is mbackup-for-windows.dll/exe
- make rsync command work.
- how to backup only documents in "My Documents" and "Downloads"?
I don't want to backup exe/msi/zip/rar/7z/iso/tar.gz files in those dir.
Only backup pdf/docx/xlsx etc.
Maybe just let user add "My Documents" in local.list.
I won't backup anything by default.
This is not good enough. docs should be backed up by default.
can I backup a zip file if I have Documents/**.zip in exclude list?
- TODO F# doesn't have a free logging framework yet?
logary/LICENSE.md at master · logary/logary · GitHub
https://github.com/logary/logary/blob/master/LICENSE.md
this one is not free software.
-
** 2019-11-12 how to test it? test it in a win 10 VM?
- search: lite weight win 10 VM
There are pirated specialized versions of Windows 7 and XP out there called
TinyXP and Tiny7. They are very stripped down and light weight.
Tiny7 is created using AIK.
https://en.wikipedia.org/wiki/Windows_Automated_Installation_Kit
- try get tiny7 iso/disk image.
tiny7 is no longer maintained.
try the old version.
Tiny 7 Windows : eXPreience : Free Download, Borrow, and Streaming : Internet Archive
https://archive.org/details/Tiny7
2017.5
- I have a 4.1G MSEdge win10 vmdk disk on sheni. from 2016.
maybe just use this.
- there is no enough RAM on B75I3. I will stop bogon VM and run win 10 VM.
-
- features
- windows installer.
bundles mingw rsync and ssh.
they are NOT installed to PATH. so it will not affect other ssh programs.
- backup file list;
user file list;
default exclude file list;
user exclude file list;
etc
They have different value compared to mbackup for linux.
- script designed to run via windows scheduled task.
no service installed. no auto starts.
taskschd.msc
- implementation.
- there is rsync running on B75I3.
exported tasksched.msc config as xml.
that XML is not portable.
it contains hostname.
-
* done :entry:
** 2019-11-16 support reading remote-user=xxx option from config file.
cli arg > env var > config file.
same with target option.
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
** 2019-11-13 next todos; scheduled task command line.
- DONE fix TODOs in F# code
- DONE add rsync command and arguments for running in windows.
i.e. -e etc.
<Arguments>--stats -togr --chown=sylecn:sylecn --exclude-from=/cygdrive/d/sylecn_docs/texts/configs/rsync-exclude
--files-from=/cygdrive/d/sylecn_docs/texts/configs/rsync-file-list
--log-file=/cygdrive/d/sylecn_docs/rsync-b75i3.log
-e ".\ssh.exe -F c:/users/sylecn/.ssh/config -i c:/Users/sylecn/.ssh/id_rsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null"
/ sheni:/data/backup/server-backup/b75i3/</Arguments>
- DONE test param parsing is working.
command line param > env var > mbackup default value.
support --node-name param.
- DONE test run in console and scheduled task.
run in console works.
try run in scheduled task.
SCHTASKS /Create /?
# run mbackup 15m after user logon.
SCHTASKS /Create /U <username> /SC ONLOGON /TN mbackup-logon /TR "\"<path\to\mbackup.exe>\" \"args\"" /DELAY 15:00
# run mbackup at 10am and 4pm.
SCHTASKS /Create /U <username> /SC DAILY /TN mbackup-morning /TR "\"<path\to\mbackup.exe>\" \"args\"" /ST 10:00 /ET 13:00 /K
SCHTASKS /Create /U <username> /SC DAILY /TN mbackup-afternoon /TR "\"<path\to\mbackup.exe>\" \"args\"" /ST 16:00 /ET 19:00 /K
# debug purpose, one time only
SCHTASKS /Create /SC ONCE /TN mbackup-test /TR "\"<path\to\mbackup.exe>\" \"args\"" /ST 19:18
in powershell,
$mbackupexe = "D:\sylecn_docs\projects\mbackup-for-windows\bin\Debug\netcoreapp3.0\mbackup-for-windows.exe"
SCHTASKS /Create /SC ONCE /TN mbackup-test /TR $mbackupexe /ST 19:21
default run as user is current user.
in admin powershell,
SCHTASKS /Create /SC ONCE /TN mbackup-test /TR $mbackupexe /ST 19:32 /NP
/NP option requires admin privilledge.
it works.
to delete it,
schtasks /delete /tn mbackup-test
- problems
- how many scheduled task to run on a multi-user PC?
each user have its own user-default.list expansion.
should I iter over all users on PC?
I think only current user can get it's profile dir and special dirs.
- maybe config mbackup to run after user logon. with 15minute delay.
always run as current user.
- how to not require any param when running mbackup.exe?
put TARGET and other option in a config file?
define system level TARGET env variable.
use config file is easier for user to edit and making the change effective.
userConfigDir / mbackup.conf
search: F# read config file
FSharp.Configuration
http://fsprojects.github.io/FSharp.Configuration/
- FSharp.Configuration missing reference to System.Runtime.Caching
search: how to reference System.Runtime.Caching for dotnet core project
https://www.nuget.org/packages/System.Runtime.Caching/
dotnet add package System.Runtime.Caching --version 4.6.0
should I add 4.0.0? Can I use a higher version?
still not compatible.
search: use FSharp.Configuration with dotnet core 3
- give up on FSharp.Configuration.
- try this:
FsConfig
https://www.demystifyfp.com/FsConfig/
AppSettings is only supported in V0.0.6 or below.
try F# AppSettings directly.
If nothing is easy to use, write my own parser.
Support similar config format as python wells lib.
- ConfigurationManager.AppSettings Property (System.Configuration) | Microsoft Docs
https://docs.microsoft.com/en-us/dotnet/api/system.configuration.configurationmanager.appsettings?view=netframework-4.8
where should I save the App.config xml file?
search: what is app.config
App.Config: Basics and Best Practices - SubMain Blog
https://blog.submain.com/app-config-basics-best-practices/
What is App.config in C#.NET? How to use it? - Stack Overflow
https://stackoverflow.com/questions/13043530/what-is-app-config-in-c-net-how-to-use-it
Okay. This is not what I want.
This is for application configuration (rarely change), not for user configuration (can change any time).
dotnet will create <yourapp>.exe.config from your App.config file.
- how to use multiple files in F# dotnet core project?
search: f# module and namespace
search: f# module
Modules - F# | Microsoft Docs
https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/modules
- SCHTASKS /Create /SC ONCE /TN mbackup-test /TR $mbackupexe /ST 19:21
this runs, it will show a console window. how to not show console window?
search: schedule tasks do not show console window
In the “General” tab, under the “Security options” section, select the Run
whether user is logged on or not option. (This is the option that will make
the command window not to appear when the task runs automatically.)
try /NP option.
- create an installer. The installer should add scheduled task on install and
delete scheduled task on removal.
- problems
- how to write unit test in F#?
Unit testing F# in .NET Core with dotnet test and NUnit - .NET Core | Microsoft Docs
https://docs.microsoft.com/en-us/dotnet/core/testing/unit-testing-fsharp-with-nunit
mkdir mbackup-tests
cd mbackup-tests
dotnet new nunit -lang F#
//wait.
dotnet test
dotnet add reference ../mbackup-for-windows.fsproj
dotnet build # this will build both ref project and test project.
dotnet test
- how to support short option names?
--dry-run -n
-
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
** 2019-11-14 support mbackup.txt config.
config file will have wells config file format.
empty lines and comment lines are ignored.
space around = is ignored.
if there are quotes on value side, it's trimmed.
target=user@host:port/path/
# target = user@host:port/path/
abc.def = some value
abc.def.ghi = 1233.45
foo = yes
env variable will be
TARGET
ABC_DEF
ABC_DEF_GHI
FOO
in F# code,
conf = WellsConfig("C:\path\to\file.txt")
conf.GetStr("target")
conf.GetFloat("abc.def.ghi")
conf.GetBool("foo")
it works.
** 2019-11-13 extra user default list.
Documents is replaced by real path.
Downloads
Pictures
User should always use forward slash in backup list and exclude list files.
These path are supported:
C:\Foo\Bar
C:/Foo/Bar
Pictures/Saved Pictures
Documents/
** 2019-11-13 build mbackup.list file from file list and exclude lists.