Commit fa27eb3a authored by Yuanle Song's avatar Yuanle Song
Browse files

v2.0.7 bugfix on zero-input-pinyin 

sometimes after GetCandidate dbus call, zero-input-fetch-size and
zero-input-candidates is not updated. This was because the async
function callback didn't update the metadata in some places.

- update initial fetch size and fetch size to be 1+ candidates needed for
  target page.
- removed zero-input-build-candidates-complete function,
  each async call should have its own callback function, don't reuse this one.
  in callback function, post GetCandidate metadata should be updated.
parent 218ef5e6

Makefile

+1 −1
Original line number Diff line number Diff line
@@ -14,7 +14,7 @@ zip: 
dist: dist-check

build:

	if [ ! -x ~/.local/bin/pytest ]; then python3 -m pip install --user pytest; fi

	~/.local/bin/pytest build.py

	python3 -m pytest build.py

	./build.py

	sed -i "s/PKG_VERSION/$(VERSION)/g" zero-input.el

dist-check: build

operational

+165 −1
Original line number Diff line number Diff line 
* COMMENT -*- mode: org -*-

#+Date: 2019-10-08

Time-stamp: <2019-11-10>

Time-stamp: <2020-02-04>

#+STARTUP: content

* notes                                                               :entry:

** 2019-04-01 zero-el a Chinese IM framework in emacs; FSM              :doc:
@@ -88,6 +88,170 @@ complete registered im symbols. 
** 2019-10-23 checkdoc and package-lint can't ignore some non-issues.

I can't run them in git pre-commit hook.

* done                                                                :entry:

** 2020-02-03 部分词组，打字时只出现词组，不能选单字。

zhiyuan 支援

zhishi 知识

weishi 微湿

weiyi 卫衣



- zhiyuan，zhishi 已修复。

  首次取词 21 个即可。

  // 不过这是假修复。没有解决词不够时，没有继续获取新词的问题。

- weishi，weiyi 还是有问题。

- all problems fixed.

- problems

  - added some debug log.

    zero-input-fetch-size is just wrong.

    "already fetched size" is wrong. which result in zero not trying to fetch

    more candidate when there is not enough to show in panel.

  - weiyi 卫衣

    50



    debug log

    #+BEGIN_SRC sh

      user typed: y

      still preediting

      appended y, preedit str is: weiy

      zero-input-pinyin building candidate list synchronously

      set zero-input-fetch-size to (max 21 (卫衣 味业 唯一 尾页 位于 未有 为已 委员 伪影 为一 惟一 唯有 卫浴 惟有 位於 违约 伟业 为由 为宜 喂养 位元 为有 威严 未央 位移 维亚 未用 胃炎 为与 为以 胃液 尾翼 为友 微一 尾牙 为用 委以 微雨 未予 为要 为伊 未遇 为盈 为业 为也 为云 威远 为月 微有 威仪)) = 50

      candidates: 卫衣, 味业, 唯一, 尾页, 位于, 未有, 为已, 委员, 伪影, 为一

      user typed: i

      still preediting

      appended i, preedit str is: weiyi

      zero-input-pinyin building candidate list synchronously

      set zero-input-fetch-size to (max 21 (卫衣 唯一 为已 为一 惟一 为宜 位移 为以 尾翼 微一 委以 为伊 威仪 偎依 逶迤 为意 维一 唯以 维以 伟易 唯壹 苇一)) = 22

      candidates: 卫衣, 唯一, 为已, 为一, 惟一, 为宜, 位移, 为以, 尾翼, 微一

      candidates: 卫衣, 唯一, 为已, 为一, 惟一, 为宜, 位移, 为以, 尾翼, 微一

      zero-input-enable-preediting-map

      user typed: =

      still preediting

      decide whether to fetch more candidates, on page 0, has 22 candidates, already fetched 50, new-fetch-size=21

      showing candidates on page 1

      candidates: 委以, 为伊, 威仪, 偎依, 逶迤, 为意, 维一, 唯以, 维以, 伟易

      candidates: 委以, 为伊, 威仪, 偎依, 逶迤, 为意, 维一, 唯以, 维以, 伟易

      zero-input-enable-preediting-map

      candidates: 委以, 为伊, 威仪, 偎依, 逶迤, 为意, 维一, 唯以, 维以, 伟易

      zero-input-enable-preediting-map

    #+END_SRC

    also, why show candidates are called twice?

  - can I trace zero-input-fetch-size variable in emacs lisp?



    replace byte-compiled version with non-byte-compiled version.



    Command: debug-on-entry function-name

    Command: cancel-debug-on-entry &optional function-name

  - I have a feeling of what's wrong.

    since the value is last time's fetch size.

    max of something, that something is not set properly.

    not zero-input-fetch-size itself.



    length of candidates. the candidates list is not updated in time or in

    correct order.



    yes. zero-input-pinyin-build-candidates doesn't set zero-input-candidates

    variable in time.



    who/when sets zero-input-candidates after fetching new candidates?



    in zero-input-framework.el, zero-input-build-candidates.



    it's override in zero-input-pinyin.el zero-input-pinyin-build-candidates.



    why zero-input-pinyin-build-candidates return the candidates as result. I

    think the return value is discarded. the return value is used by async

    complete function, used in default async implementation.



    how to distinguish between async and sync fetch?

    in async fetch, should update zero-input-candidates when fetch finished.

    in sync fetch, should update zero-input-candidates right away.

    I don't need to distinguish them, if zero-input-pinyin-build-candidates is

    called, it is using sync fetch. There is no time delay or async io.

  - DONE when is sync version used, when is async version used?

    // sync version is always called via async-func.



    zero-input-pinyin-page-down always call async verison.

    it relies on zero-input-build-candidates-complete to set

    zero-input-candidates variable.



    - when preedit-str changes.

      zero-input-preedit-str-changed

      calls zero-input-build-candidates-async-func



    - when user press = goto next page.

      zero-input-page-down

      calls zero-input-build-candidates-async-func



    - so sync version is always called via async-func.



    - zero-input-build-candidates-complete is not used properly in

      zero-input-page-down. do not reuse zero-input-build-candidates-complete

      function, just inline what should be called.



      #+BEGIN_SRC emacs-lisp

        (defun zero-input-build-candidates-complete (candidates)

          "Called when `zero-input-build-candidates-async' return.



        CANDIDATES is returned candidates list from async call."

          (setq zero-input-candidates candidates)

          (zero-input-show-candidates candidates))

      #+END_SRC

      this is where two show-candidates calls are made.

      now fixed.



    - double check zero-input-pinyin-build-candidates-async doesn't do the

      same thing that the callback function already did.



      it has done it twice.

      fixed.



    - zero-input-framework.el sync/async version is not quite right.

      zero-input-pinyin.el async version is not being used? It's okay.

      does sync version work as expected?



      the two "external" use case always call async version, so sync version

      is only used to build candidates, and I confirm it doesn't need to

      update metadata.



      zero pinyin:

      #+BEGIN_SRC emacs-lisp

        (zero-input-register-im

         'pinyin

         '((:build-candidates . zero-input-pinyin-build-candidates)

           ;; comment to use sync version, uncomment to use async version.

           ;; (:build-candidates-async . zero-input-pinyin-build-candidates-async)

      #+END_SRC



    - debug log.



      #+BEGIN_SRC sh

        zero-input-pinyin building candidate list synchronously

        zero-input-pinyin building candidate list asynchronously

      #+END_SRC



      zero-input-pinyin.el why sometimes it uses sync call and sometimes it

      uses async call? I think every async call should chain back to sync

      call.



      I think when a call is triggered from zero-framework.el, it will always

      be sync call. when a call is triggered from zero-pinyin.el, it MAY used

      async call directly.



      who calls zero-input-pinyin-build-candidates-async



      - zero-input-pinyin-preedit-str-changed

	calls framework's version. which use default async call. and chains to

        zero-pinyin sync call.

      - zero-input-pinyin-pending-preedit-str-changed

	uses async call.

      - zero-input-pinyin-page-down

	uses async call.



      No wonder zero-pinyin sync and async performance is on par. Because sync

      call is not used much.

    - How to use sync or async call in all places? I don't want to mix it.

      just enable async in IM register code?

    - 



** 2019-11-10 space doesn't work in full-width mode.

全角和半角 - 维基百科，自由的百科全书

https://zh.wikipedia.org/wiki/%E5%85%A8%E5%BD%A2%E5%92%8C%E5%8D%8A%E5%BD%A2

zero-input-framework-test.el

+14 −0
Original line number Diff line number Diff line
@@ -40,6 +40,20 @@ 
  (should (string-equal "ｈｅｈｅ" (zero-input-convert-str-to-full-width "hehe")))

  (should (string-equal "（Ａ）" (zero-input-convert-str-to-full-width "(A)"))))



(ert-deftest zero-input-get-initial-fetch-size ()

  (let ((zero-input-initial-fetch-size 20))

    (should (= 21 (zero-input-get-initial-fetch-size))))

  (let ((zero-input-initial-fetch-size 19))

    (should (= 19 (zero-input-get-initial-fetch-size))))

  (let ((zero-input-initial-fetch-size 9))

    (should (= 11 (zero-input-get-initial-fetch-size))))

  (let ((zero-input-initial-fetch-size 10))

    (should (= 11 (zero-input-get-initial-fetch-size))))

  (let ((zero-input-initial-fetch-size 11))

    (should (= 11 (zero-input-get-initial-fetch-size))))

  (let ((zero-input-initial-fetch-size 12))

    (should (= 12 (zero-input-get-initial-fetch-size)))))



(provide 'zero-input-framework-test)



;;; zero-input-framework-test.el ends here

zero-input-framework.el

+47 −27
Original line number Diff line number Diff line
@@ -132,7 +132,7 @@ If item is not in lst, return nil." 


;; zero-input-el version

(defvar zero-input-version nil "Zero package version.")

(setq zero-input-version "2.0.6")

(setq zero-input-version "2.0.7")



;; FSM state

(defconst zero-input--state-im-off 'IM-OFF)
@@ -195,11 +195,14 @@ Change will be effective only in new `zero-input-mode' buffer." 
  :group 'zero

  :type 'integer)

(defvar-local zero-input-current-page 0 "Current page number.  count from 0.")

(defvar-local zero-input-initial-fetch-size 20

  "How many candidates to fetch for the first call to GetCandidates.")

(defvar-local zero-input-initial-fetch-size 21

  "How many candidates to fetch for the first call to GetCandidates.



It's best set to (1+ (* zero-input-candidates-per-page N)) where

N is number of pages you want to fetch in initial fetch.")

;; zero-input-fetch-size is reset to 0 when preedit-str changes.

;; zero-input-fetch-size is set to fetch-size in build-candidates-async complete-func

;; lambda.

;; zero-input-fetch-size is set to fetch-size in build-candidates-async

;; complete-func lambda.

(defvar-local zero-input-fetch-size 0 "Last GetCandidates call's fetch-size.")

(defvar zero-input-previous-page-key ?\- "Previous page key.")

(defvar zero-input-next-page-key ?\= "Next page key.")
@@ -308,7 +311,8 @@ enough elements, return lst as it is." 


(defun zero-input-show-candidates (&optional candidates)

  "Show CANDIDATES using zero-input-panel via IPC/RPC."

  (let ((candidates-on-page (zero-input-candidates-on-page (or candidates

  (let ((candidates-on-page

	 (zero-input-candidates-on-page (or candidates

					    zero-input-candidates))))

    (cl-destructuring-bind (x y) (zero-input-get-point-position)

      (zero-input-panel-show-candidates
@@ -327,19 +331,15 @@ enough elements, return lst as it is." 
(defun zero-input-build-candidates (preedit-str fetch-size)

  "Build candidates list synchronously.



Try to find at least FETCH-SIZE number of candidates for PREEDIT-STR."

Try to find at least FETCH-SIZE number of candidates for PREEDIT-STR.

Return a list of candidates."

  ;; (zero-input-debug "zero-input-build-candidates\n")

  (unless (functionp zero-input-build-candidates-func)

    (signal 'wrong-type-argument (list 'functionp zero-input-build-candidates-func)))

  (prog1 (funcall zero-input-build-candidates-func preedit-str fetch-size)

    (setq zero-input-fetch-size (max fetch-size (length zero-input-candidates)))))



(defun zero-input-build-candidates-complete (candidates)

  "Called when `zero-input-build-candidates-async' return.



CANDIDATES is returned candidates list from async call."

  (setq zero-input-candidates candidates)

  (zero-input-show-candidates candidates))

  ;; Note that zero-input-candidates and zero-input-fetch-size is updated in

  ;; async complete callback. This function only care about building the

  ;; candidates.

  (funcall zero-input-build-candidates-func preedit-str fetch-size))



(defun zero-input-build-candidates-async-default (preedit-str fetch-size complete-func)

  "Build candidate list, when done show it via `zero-input-show-candidates'.
@@ -348,6 +348,7 @@ PREEDIT-STR the preedit-str. 
FETCH-SIZE try to find at least this many candidates for preedit-str.

COMPLETE-FUNC the function to call when build candidates completes."

  ;; (zero-input-debug "zero-input-build-candidates-async-default\n")

  ;; default implementation just call sync version `zero-input-build-candidates'.

  (let ((candidates (zero-input-build-candidates preedit-str fetch-size)))

    ;; update cache to make SPC and digit key selection possible.

    (funcall complete-func candidates)))
@@ -463,23 +464,27 @@ Return CH's Chinese punctuation if CH is converted. Return nil otherwise." 
  (let ((len (length zero-input-candidates)))

    (when (> len (* zero-input-candidates-per-page (1+ zero-input-current-page)))

      (setq zero-input-current-page (1+ zero-input-current-page))

      (zero-input-debug "showing candidates on page %s\n" zero-input-current-page)

      (zero-input-show-candidates))))



(defun zero-input-page-down ()

  "If there is still candidates to be displayed, show candidates on next page."

  (interactive)

  (let ((len (length zero-input-candidates))

	(new-fetch-size (* zero-input-candidates-per-page (+ 2 zero-input-current-page))))

	(new-fetch-size (1+ (* zero-input-candidates-per-page (+ 2 zero-input-current-page)))))

    (zero-input-debug "decide whether to fetch more candidates, has %s candidates, last fetch size=%s, new-fetch-size=%s\n" len zero-input-fetch-size new-fetch-size)

    (if (and (< len new-fetch-size)

	     (< zero-input-fetch-size new-fetch-size))

	(progn

	  (zero-input-debug "will fetch more candidates")

	  (funcall zero-input-build-candidates-async-func

		   zero-input-preedit-str

		   new-fetch-size

		   (lambda (candidates)

		   (zero-input-build-candidates-complete candidates)

		     (setq zero-input-candidates candidates)

		     (setq zero-input-fetch-size (max new-fetch-size

						      (length candidates)))

		   (zero-input-just-page-down)))

		     (zero-input-just-page-down))))

      (zero-input-just-page-down))))



(defun zero-input-handle-preedit-char-default (ch)
@@ -533,11 +538,26 @@ N is the argument passed to `self-insert-command'." 
      (zero-input-debug "unexpected state: %s\n" zero-input-state)

      (self-insert-command n)))))



(defun zero-input-get-initial-fetch-size ()

  "return initial fetch size"

  (cond

   ((<= zero-input-initial-fetch-size zero-input-candidates-per-page)

    (1+ zero-input-candidates-per-page))

   ((zerop (mod zero-input-initial-fetch-size zero-input-candidates-per-page))

    (1+ zero-input-initial-fetch-size))

   (t zero-input-initial-fetch-size)))



(defun zero-input-preedit-str-changed ()

  "Called when preedit str is changed and not empty.  Update and show candidate list."

  (setq zero-input-fetch-size 0)

  (setq zero-input-current-page 0)

  (funcall zero-input-build-candidates-async-func zero-input-preedit-str zero-input-initial-fetch-size 'zero-input-build-candidates-complete))

  (let ((new-fetch-size (zero-input-get-initial-fetch-size)))

    (funcall zero-input-build-candidates-async-func

	     zero-input-preedit-str new-fetch-size

	     (lambda (candidates)

	       (setq zero-input-candidates candidates)

	       (setq zero-input-fetch-size (max new-fetch-size (length candidates)))

	       (zero-input-show-candidates candidates)))))



(defun zero-input-backspace-default ()

  "Handle backspace key in `zero-input--state-im-preediting' state."

zero-input-pinyin.el

+26 −11
Original line number Diff line number Diff line
@@ -100,7 +100,9 @@ PREEDIT-STR the preedit string. 
FETCH-SIZE fetch at least this many candidates if possible."

  (zero-input-debug "zero-input-pinyin building candidate list synchronously\n")

  (let ((result (zero-input-pinyin-service-get-candidates preedit-str fetch-size)))

    (setq zero-input-fetch-size (max fetch-size (length (cl-first result))))

    ;; update zero-input-candidates and zero-input-fetch-size is done in async

    ;; complete callback. This function only care about building the

    ;; candidates and updating zero-input-pinyin specific.

    (setq zero-input-pinyin-used-preedit-str-lengths (cl-second result))

    (setq zero-input-pinyin-candidates-pinyin-indices (cl-third result))

    (cl-first result)))
@@ -117,9 +119,10 @@ COMPLETE-FUNC the callback function when async call completes. it's called with 
   preedit-str

   fetch-size

   (lambda (candidates matched_preedit_str_lengths candidates_pinyin_indices)

     (setq zero-input-candidates candidates)

     (setq zero-input-fetch-size (max fetch-size (length candidates)))

     (setq zero-input-pinyin-used-preedit-str-lengths matched_preedit_str_lengths)

     (setq zero-input-pinyin-candidates-pinyin-indices candidates_pinyin_indices)

     (setq zero-input-fetch-size (max fetch-size (length candidates)))

     ;; Note: with dynamic binding, this command result in (void-variable

     ;; complete-func) error.

     (funcall complete-func candidates))))
@@ -136,7 +139,10 @@ COMPLETE-FUNC the callback function when async call completes. it's called with 
  "Update zero states when pending preedit string changed."

  (setq zero-input-fetch-size 0)

  (setq zero-input-current-page 0)

  (zero-input-pinyin-build-candidates-async zero-input-pinyin-pending-preedit-str zero-input-initial-fetch-size 'zero-input-build-candidates-complete))

  (let ((new-fetch-size (zero-input-get-initial-fetch-size)))

    (zero-input-pinyin-build-candidates-async

     zero-input-pinyin-pending-preedit-str new-fetch-size

     #'zero-input-show-candidates)))



(defun zero-input-pinyin-commit-nth-candidate (n)

  "Commit Nth candidate and return true if it exists, otherwise, return false."
@@ -222,16 +228,25 @@ Otherwise, just return nil." 


This is different from zero-input-framework because I need to support partial commit"

  (let ((len (length zero-input-candidates))

	(new-fetch-size (* zero-input-candidates-per-page (+ 2 zero-input-current-page))))

	(new-fetch-size (1+ (* zero-input-candidates-per-page (+ 2 zero-input-current-page)))))

    (zero-input-debug

     "fetch more candidates? on page %s, has %s candidates, last-fetch-size=%s, new-fetch-size=%s\n"

     zero-input-current-page len zero-input-fetch-size new-fetch-size)

    (if (and (< len new-fetch-size)

	     (< zero-input-fetch-size new-fetch-size))

	(let ((preedit-str (if (eq zero-input-pinyin-state zero-input-pinyin--state-im-partial-commit) zero-input-pinyin-pending-preedit-str zero-input-preedit-str)))

	(progn

	  (zero-input-debug

	   "will fetch more candidates new-fetch-size=%s\n" new-fetch-size)

	  (let ((preedit-str (if (eq zero-input-pinyin-state

				     zero-input-pinyin--state-im-partial-commit)

				 zero-input-pinyin-pending-preedit-str

			       zero-input-preedit-str)))

	    (zero-input-pinyin-build-candidates-async

	     preedit-str

	     new-fetch-size

	   (lambda (candidates)

	     (zero-input-build-candidates-complete candidates)

	     (zero-input-just-page-down))))

	     (lambda (_candidates)

	       (zero-input-just-page-down)))))

      (zero-input-debug "won't fetch more candidates\n")

      (zero-input-just-page-down))))



(defun zero-input-pinyin-handle-preedit-char (ch)