operational

* COMMENT -*- mode: org -*-
#+Date: 2015-11-04
Time-stamp: <2015-12-31>
#+STARTUP: content
* later								      :entry:
** 2015-11-04 warn when hypervisor can't start KVM type VM.
* current							      :entry:
** 
** 2015-12-31 a new design of qvm (continue)
Here are things not implemented yet.
*** qvm create
    - supported disk type

      --disk /path/to/foo.img
      --disk ubuntu-14.04    find file with this name in images dir.
      --disk sheepdog:foo    clone from this sheepdog snapshot/disk.
      --disk URL-to-img-file download this file. use local copy if it exists.
    - supported network

      --network type=bridge,bridge=br1,vlan=200,ip=192.168.1.2
      --network type=nat,bridge=br1,ip=192.168.1.2

      omit this parameter means use defined:default network.

      --network predefined:default
      --network predefined:ovs1
      --network ovsbridge:name=br0
      --network ovsbridge:name=br0,vlan=200
*** support bind IP using metadata service.
*** support resize disk to given size.
    qemu-img resize foo.img 20G
*** auto detect and convert raw type disk to qcow2 type.

** 2015-11-04 how to ensure VM get same IP across reboots?
how to list IPs for all VMs?

* done								      :entry:
** 2015-12-31 if any of disk, domain.xml exists, just exit early.
** 2015-12-30 a new design of qvm.
- make create ubuntu VM easy. using qvm and sheepdog/nginx.
  - qvm init
    prepare env for HV.
    - install libvirt, qemu, ovs, kvm-check etc.
    - create ovs network or bridge. define libvirt network.
    - [optional] create secrets for ceph.
  - qvm create vm1 --core 2 --ram 512M --network default --disk /path/to/foo.img

    - create image for the vm.
    - define and start the vm.

    - supported disk type

      --disk /path/to/foo.img
      --disk ubuntu-14.04    find file with this name in images dir.
      --disk sheepdog:foo    clone from this sheepdog snapshot/disk.
      --disk URL-to-img-file download this file. use local copy if it exists.

      When download from URL, just save the file using the name of sha1 of
      URL, with original file suffix (for easier qemu-img operation).

      They all means make a copy of the image/snapshot and use that as first
      disk. Resize it to user requested size when cloning.

      - DONE URL type image.
	:t toLazyByteString $ byteStringHex $ SHA1.hash $ B.pack "abc"

	- how to download large file in haskell without using much RAM.
	  http://stackoverflow.com/questions/24718873/downloading-large-files-from-the-internet-in-haskell
	  Network.HTTP.Conduit http can do it.

	  https://hackage.haskell.org/package/download-curl-0.1.4/docs/Network-Curl-Download-Lazy.html
	  I hope openLazyURI can do it too. Because it's easier.
	  download-curl is not in lts haskell.
	  add it in stack.yaml. it works.

    - supported network

      --network type=bridge,bridge=br1,vlan=200,ip=192.168.1.2
      --network type=nat,bridge=br1,ip=192.168.1.2

      omit this parameter means use defined:default network.

      --network predefined:default
      --network predefined:ovs1
      --network ovsbridge:name=br0
      --network ovsbridge:name=br0,vlan=200

  - qvm rm vm1

    destroy domain, undefine domain and remove disk and xml file.

  - implementation

    - DONE base image naming schema. I want to include original name after sha1.
    - WONTFIX generate a random name if user didn't specify a name.
      error if user didn't specify a name.
      when running from cmd line, user have to specify a name.
      - check whether VM name already exists before doing anything.

    - DONE make nat and file base image work.

      - DONE where to do disk copy action?
	when you `create' the domain. not just building the domain.
      - DONE implementation renderDisk.
	should I support both qcow2 and raw? only support qcow2 for now.
	what is bochs and qed?

	- DONE can I resize when clone from qcow2?

	  Just copy it, then resize.
          qemu-img resize foo.img 20G

	- abort if disk is not in qcow2 format.
	- DONE is it easy to share a base when using qcow2? backing file?

	  yes. do not use copy file, use qemu-img:

	  qemu-img create -f qcow2 -b centos-cleaninstall.img snapshot.img

	  The new image is now a read/write snapshot of the original image --
          any changes to snapshot.img will not be reflected in
          centos-cleaninstall.img.

	  OK. I will only support qcow2 format disk.
	  Show error if given disk file is not qcow2.
          TODO I can also convert it automatically and use that as base.

    - DONE implement "qvm init"
      I don't like the way haskell handle cli arguments.
      But for type safety, they have to be more complicated.

      I want my program to be self contained. so I don't plan to run
      puppet/salt there.

      DONE qvmInit :: IO ()

      DONE support "qvm init" in cmd line handling.

    - DONE how to use relative file path in stFile?
      renderXML d = [stFile|src/domain-templates/domain.xml|]
      works. just need src/ there. build always happens in stack.yaml dir.

    - DONE use text file template to render xml.
      network and disk should be placed in as a whole.

      Which library to use?
      libraries - Which Haskell library for interpolated strings - Stack Overflow
      http://stackoverflow.com/questions/8956801/which-haskell-library-for-interpolated-strings

      try shakespeare
      http://hackage.haskell.org/package/shakespeare

      render to Text using quasiquotes works.
      #+BEGIN_SRC haskell
        renderXML :: QvmDomain -> T.Text
        renderXML d = [st|
        haha my core is #{core d}
        |]
      #+END_SRC

      TODO how to render using external file?
      I think the template file is compiled into the final binary.

      compare it with lucius.

      hamlet :: QuasiQuoter
      hamletFile :: FilePath -> Q Exp

      putStrLn $ renderHtml $ [hamlet|<body>|] render

      template = $(luciusFile "template.lucius")
      putStrLn $ renderCss $ template render

      http://stackoverflow.com/questions/12791491/use-shakespeare-text-and-external-file
      didn't mention a solution.

      - I need to understand template haskell.

        what does quasiquoter produce?
        how to use Q Exp?

	understand what st is doing, then do the similar for textFile.
	http://hackage.haskell.org/package/shakespeare-2.0.7/docs/src/Text-Shakespeare-Text.html#stext

	#+BEGIN_SRC haskell
          stext :: QuasiQuoter
          stext = 
            QuasiQuoter { quoteExp = \s -> do
              rs <- settings
              render <- [|toLazyText|]
              rendered <- shakespeareFromString rs { justVarInterpolation = True } s
              return (render `AppE` rendered)
              }

          textFile :: FilePath -> Q Exp
          textFile fp = do
              rs <- settings
              shakespeareFile rs fp
	#+END_SRC

	QuasiQuoter {quoteExp :: String -> Q Exp}

	shakespeareFromString :: ShakespeareSettings -> String -> Q Exp
	shakespeareFile :: ShakespeareSettings -> FilePath -> Q Exp

      - copied some code and it is working now.

      - shift2space for pretty xml.

        TIO.putStrLn $ shift2space $ renderNetwork $ DefaultNAT
	printXML makeDomain

    - DONE OO design or functional? I like functional.

      do
        makeDomain
        setCore 2
        setRAM "512M"
        setNetwork {type: "nat", bridge: "br1"}
        setDisk {src: "/path/to/foo.img"}
        printDomainXML

      store all useful info in data structure, then generate xml in the final
      step. do not do xml level combination/replacing.

    - DONE qvm config directory.
      /data/qvm/images/
      /data/qvm/images/cirros-0.3.4-x86_64-disk.img
      /data/qvm/instances/
      /data/qvm/instances/vm1/disk
      /data/qvm/instances/vm1/domain.xml

    - DONE network type
      predefined nat. params: network name

      #+BEGIN_SRC xml
        <interface type='network'>
          <source network='default'/>
          <model type='virtio'/>
        </interface>
      #+END_SRC

      bridge mode with vlan tag. params: bridge, vlanid
      #+BEGIN_SRC xml
        <interface type='bridge'>
          <forward mode="bridge"/>
          <source bridge="br1"/>
          <virtualport type='openvswitch'/>
          <model type='virtio'/>
          <vlan trunk='no'>
            <tag id='100'/>
          </vlan>
        </interface>
      #+END_SRC

      putStrLn $ T.unpack $ renderNetwork DefaultNAT
      putStrLn $ T.unpack $ renderNetwork $ OvsBridge "br1"
      putStrLn $ T.unpack $ renderNetwork $ OvsBridgeWithVLAN "br1" 100

* wontfix							      :entry:
** 2015-11-04 make this work:
qvm --core 4 --ram 2g --disk 20g

use NAT network and default disk image.

- check whether we have enough RAM for this.
- generate xml
- define domain
- start domain