Freezing when exiting insert mode in large files. about vimtex

[tbrosnan]

Here is some profiling information. Procedure was:
-open file
-start profiling
-enter insert mode
-exit insert mode
-stop profiling

FUNCTION  latex#fold#level()
Called 342 times
Total time:   9.250681
 Self time:   9.250681

count  total (s)   self (s)
                              " Check for normal lines first (optimization)
  342              0.002613   let line  = getline(a:lnum)
  342              0.017526   if line !~ '\(% Fake\|\\\(document\|begin\|end\|' . 'front\|main\|back\|app\|sub\|section\|chapter\|part\)\)'
  245              0.001003     return "="
                              endif

                              " Fold preamble
   97              0.000378   if g:latex_fold_preamble
   97              0.001407     if line =~# '^\s*\\documentclass'
                                  return ">1"
                                elseif line =~# '^\s*\\begin\s*{\s*document\s*}'
                                  return "0"
                                endif
   97              0.000247   endif

                              " Fold chapters and sections
  411              0.002010   for [part, level] in b:latex.fold_parts
  339              0.005265     if line =~# part
   25              0.000208       return ">" . level
                                endif
  314              0.000833   endfor

                              " Never fold \end{document}
   72              0.001017   if line =~# '^\s*\\end{document}'
                                return 0
                              endif

                              " Fold environments
   72              0.000247   if g:latex_fold_envs
   72              4.415269     if line =~# s:notcomment . s:notbslash . '\\begin\s*{.\{-}}'
   33              0.000325       if line !~# '\\end'
   33              0.000122         return "a1"
                                  endif
                                elseif line =~# s:notcomment . s:notbslash . '\\end\s*{.\{-}}'
   33              0.000332       if line !~# '\\begin'
   33              0.000142         return "s1"
                                  endif
                                endif
    6              0.000021   endif

                              " Return foldlevel of previous line
    6              0.000020   return "="

Looks like determining fold levels can be slow, possibly those regexs don't scale well to long lines. The bulk of the time is spent on matching if line =~# s:notcomment . s:notbslash . '\\begin\s*{.\{-}}'.

After set nofoldenable and let g:latex_fold_enabled=0 and reloading the file things are usable again, just without folding support.

[lervag]

I am aware of this issue, and I unfortunately don't see any good way of solving it. The issue is not with large documents, but with long lines, and it might be solved with a more clever regexp. I've tried and failed a couple of times, so I've decided not to solve this particular issue. If you are able to improve the regexp(s) and still let them match the same items, then I would be very happy for pull requests!

In the mean time, I've implemented an alternative solution that does not automatically recalculate folds, see h g:latex_fold_automatic.

Since I will not work on solving the actual problem, I will take the liberty to close the issue.

[tbrosnan]

I don't think I can contribute to this in the near future I'm afraid...

What is the purpose of the \{-} in \\begin\s*{.\{-}}? Is it to catch nested \begin{} statements on the same line?

[lervag]

Good question. It seems superfluous, and it is probably one of the main reasons that the regexp is slow. I'll take a look at it when I get the time.

[lervag]

I was looking in to this now, but I was not able to reproduce the issue. Could you send me an example where you experience the issue?

[Konfekt]

@tbrosnan .. as a preliminary workaround try my FastFold plugin.

[lervag]

@Konfekt I think that your FastFold plugin might do (approximately) the same thing as what happens with g:latex_fold_automatic = 0.

Btw: I'll close the issue for now. Feel free to reopen if you deem it necessary.

Edit: Corrected value of g:latex_fold_automatic from 1 to 0.

[Konfekt]

Ok, by the issue title seemingly folds would be continuously updated when leaving insert mode, but when they are only updated once on zx it is similar to what FasFold does.

[lervag]

Yes, the default mode is g:latex_fold_automatic = 1, which imples that the folds are updated after leaving insert mode. If one instead uses g:latex_fold_automatic = 0, then the folds are only updated when one issues zx or zX.

[mihaelS]

Hello!
Seems vimtex#fold#level() is very slow..
in vimrc I set g:latex_fold_automatic = 0, but it doesn't help.
Is there some work around?

FUNCTIONS SORTED ON SELF TIME
count  total (s)   self (s)  function
128220             19.237260  vimtex#fold#level()
  922              0.028393  airline#highlighter#exec()
    7   0.094512   0.026499  airline#highlighter#highlight()
  756              0.019257  <SNR>162_get_syn()
    2              0.014834  <SNR>153_check_mixed_indent()

[lervag]

First, all vimtex options now start with g:vimtex_....

Could you please provide the full set of options you are using? With g:vimtex_fold_automatic = 0, which is now the default value, the fold level function should only be run by the user manually through zx.

Which version of vimtex are you using?

[mihaelS]

1. It slows down when I'm printing text in insert mode.
2. It is slow only when I'm editing tex file and there is other 5 large tex files is opened in a buffers. I open all files from a session file (:mks tex_files.mks; vim -S tex_files_mks)
3. It is working pretty fast, when I'm openning a single tex file.
4. here is vim settings in the slow file https://gist.github.com/mihaelS/32bc3b1127f9ff15567a
5. I think there is the last version (current) I'm updating regulary.
   The last lines in the chenge log:
   2015-06-06: Minor but convenient restructuring (++)~
   I've changed a lot of the code structure in relatively small ways. For
   instance, instead of referring to the particular data blobs through the global
   array, I instead linked a buffer variable to the correct global array element.
   ...

[lervag]

Based on your settings you do not set any options for vimtex. Note that your vimrc file is quite messy and difficult to parse visually. Also, I see you add plugins manually, instead of through a plugin manager. This might work as expected, but it might also not. For instance, I would recommend you instead do:

let &rtp = '~/.vim/plugged/<bundle>,' . &rtp

for each <bundle> or plugin. Also, vim-latex changed the name to vimtex several months ago. I guess github still redirects vim-latex automatically to vimtex, but I don't know how long that works. And I notice you add both vim-latex and vim-latex/after. The latter is not necessary, as it is implied by the former.

What happens if you open the tex files in a fresh session (not loaded from a session file)?

Which version of Vim are you using?

[mihaelS]

I'm using 'vim-plug' pluggin manager, so it install the vimtex. So I changed:
Plug 'lervag/vimtex'
Seems that it is installed correctly.

When I'm using fresh session, it just work fast. As way as it should. No hang up after every tap on the keyboard .
1)After deliting all other buffers in the old session text inset is still slow.
2) After reload .vimrc in the old session text insert is still slow.

vim version is 7.4.52

[lervag]

What happens if you create a new session similar to your old session?

I can not say why it is slow in the old session, but my suspicion is that the old session somehow still uses the automatic folding, regardless of the contents in your vimrc file.

[mihaelS]

I made several tests:

1. open tex files in the NewSession1 in the old order. Inserting text is slow.
2. open tex files in the NewSession2 in desc order. Inserting text is fast. Then I save session. And open it. Inserting text is slow.

Result:
in the some files (not all) inserting text is critically slow.
It appears in the files with more than 1000 lines (1000, 3000, 4000). Probably it depends on the files structure..

[lervag]

This is very strange. I don't really understand what is happening here.

So, if you open a single tex file, regardless of size, everything will be fast as expected. Right? The problem here is only when you open the files and use the session feature?

[mihaelS]

yes, working in a single file is fast.
I opened 2 most big files and save the session. Session was fast.
After adding the 3rd file the big one files begin to slow.
Lags appears in some lines positions - closer to the end,  the greater the lags.

I'm addiding the files in the new tabs.

[lervag]

Could you try again in a single file, but this time make sure it is a file with very long lines? The automatic folding feature is known to become very slow on long lines (which is one of the reasons I don't recommend that option and why it is not default).

Would you be able/willing to create a minimal working example and a minimal vimrc file that makes it possible for me to reproduce this, along with a description of how to reproduce? The minimal vimrc file should be something like this:

set nocompatible
let &rtp = '~/.vim/plugged/vimtex,' . &rtp
filetype plugin indent on
syntax enable
" Add relevant vimtex options (if any)

If you are not able to reproduce the problem with a minimal example, then it would also work if you could share some files where the problem occurs.

[mihaelS]

Going to do it. I will post the results a bit later

[lahwaacz]

Some time ago, I also noticed problems with vimtex's folding and the :mksession command. Most notably, the state of the folds was not preserved, unlike other filetypes. I did not notice any slowdown in the restored session, so the problem might have been introduced in recent (~1 month) changes.

[lervag]

The last essential update was to parse modeline settings in early July. Before that I did some minor restructuring and similar things, but nothing that should change the fold feature directly. Of course, I may have introduced a bug.

In any case, until I have a clear issue that I can reproduce, I really can not help very much.

[mihaelS]

Hello!

here is an example :
http://fileload.info/y6c256j403c8

Steps to reproduce it:

1. install vim-plug :

curl -fLo ~/.vim/autoload/plug.vim --create-dirs \
https://raw.githubusercontent.com/junegunn/vim-plug/master/plug.vim

2. copy .vimrc from archive to homeDir

3. install pugins (in vim):

   :PlugInstall

4. open test.tex files from archive in separate tabs

   vim -p test*tex

5. save session (in vim):

   :mks m.mks

6. open session file

   vim -S m.mks

7. try to type text (several sentences) at the end of the .tex files above line

   \end{document} 

In some files there is lags.

[mihaelS]

Did the example is reproduced?

[lervag]

Yes, the example is reproducible. That is, I skipped the session part, steps 5 and 6, and I still verify the problem. I use the following minimal vimrc file when reproducing:

set nocompatible
let &rtp = '~/.vim/bundle/vimtex,' . &rtp
filetype plugin indent on
syntax enable

Then to reproduce, I open the tex files with vim -u minivimrc -p test*tex. Now I go to the last line and insert above the \begin{document}, and I experience the lag you are talking about.

My test indicates that the session is not relevant.

I tested further, and it seems to me that the tab window part is relevant. If I instead open with vim -u minivimrc test*.tex, where minivimrc also has set hidden, I do not experience the lag.

It would be helpful to know if my investigation matches your results so far.

[lervag]

Could I ask: The tex file seems very strange to me and has a lot of very strange signs, e.g. as shown below:

\documentclass[a4paper######,######]###{######}###
###%\usepackage{ifthen}######
\usepackage[utf8]{inputenc}######
\usepackage{gensymb}######

What are all the # signs for? Did you scramble the tex files before sharing?

[lervag]

I think I found the problem. It seems that when you open several files in tab pages, all of the buffers are loaded at once. For the manual fold mode, there is an autocommand that is created in order to ensure that the &foldmethod is manual; the autocommand self destructs when executed. Thus for the second buffer that is loaded, this autocommand has already self destructed and does not work.

I will try to find a fix.

[lervag]

Should be fixed now, please test.

[mihaelS]

Yes, I scrumble the file before posting.
Unfortunately, problem remains. Same example.

vimtex/autoload/vimtex/fold.vim:

    if g:vimtex_fold_automatic
      augroup vimtex_fold_automatic
        autocmd!
        autocmd InsertEnter <buffer> call FdmSave()
        autocmd InsertLeave <buffer> call FdmRestore()
      augroup END
    endif
  else
    augroup vimtex_fold_manual
      autocmd CursorMoved <buffer> call vimtex#fold#refresh('zx')
      autocmd CursorMoved <buffer> autocmd! vimtex_fold_manual
    augroup END

Results:

FUNCTIONS SORTED ON SELF TIME
count  total (s)   self (s)  function
111020             16.489066  vimtex#fold#level()
    2              0.013080  <SNR>165_check_mixed_indent()
  280              0.008321  <SNR>174_get_syn()
  250              0.007772  airline#highlighter#exec()
    3   0.031025   0.006955  airline#highlighter#highlight()

[lervag]

Interesting. I think I found the issue, but to be sure: Could you please remove the vimtex_fold_manual autocommand group and instead have only

  else
    setlocal fdm=manual
  endif

I think this should remove the lag, but it will introduce a second problem: the folds are not activated properly. The point of the autocommand group is to ensure that the folds are refreshed upon startup.

I think I should investigate some more and find an alternative way to ensure that the folds are refreshed.

[mihaelS]

Yes, now it pretty fast! 👍
But, folds crashed - I can not collapse the folds.

[lervag]

I've opened a question on vi.stackexchange.com.

I also think I have a new solution that should work.

[lervag]

With the fix I mentioned you have to manually refresh the folds with zx. Then the folds should be collapsable.

[mihaelS]

Yes, got it. Thx !

[lervag]

Now it should be fixed. Please test.

[mihaelS]

Works perfect! 💯

[lervag]

Great! Thanks for noticing and reporting the issue!