memory leak

[yoramo1]

I'm a big fan of this application. but i encountered a very annoying bug.
using version 1.6.6. if I leave LogExpert open for a log time it increases the memory size used and eventually will make the computer work very slow.

thanks

[Hirogen]

this is a Bug I encountered in all Versions

Can be easily retested with a continous logfile that is written while it is viewed in logexpert.

Logexpert will eventually use all the computers memory (in my case 15gigs+). After all the memory is used, the computer will be unresponsive.

Workaround is to restart it regularly or don't let it run overnight!

[zarunbal]

Thanks for the issue. I'll try to reproduce this problem.

As far as I understand the application grows if a line is appended?

[yoramo1]

I do not have a reproducible scenario for this problem.

it looks related to appending lines to the log file.

after the problem LogExpert can not be closed you need to kill it from the task manager.

[1inirt1]

I totally agree that LogExpert is awesome and I love using it. Except for this one memory leak issue. We had a test system crater over the weekend because someone left LogExpert open to a rolling log file that gets updated at least once every 15 seconds.

zarunbal, this bug is pretty easy to replicate so if you'd like exact steps to replicate please let me know because the sooner this bug can be resolved the sooner my colleagues and I can replace a whole bunch of different log viewer programs with LogExpert once and for all. Thanks and keep up the great work.

[zarunbal]

I can reproduce this bug, and already added some improvements, which lowers the increase of memory per time / line. But to find / fix the root of the problem this is still some work (despite that the problem is reproducible).

There would be a possibility for a workaround of this memory problem. I could create a 32 bit only application (it would still work on a 64 bit machine but could only address around 2 GB of ram). If the application reach the limit it will crash. This would be a rough workaround but it would prevent that LogExpert uses all the memory.

@1inirt1 thanks for your feedback. I understand your problem and the related problems for your. I work at this bug as hard as I can but this is only in my freetime.

[fremag]

With dotMemory (profiler) I looked at this issue.
I ran LogExpert on a file that was updated 20 times /secondes.
After a few hours, the memory used was around 600 Mo.
I think it should not be that high. There are some buffers for each file watched but it's limited to 10 x 100 lines. Even if a line is around 1 ko, the corresponding memory is around 1 Mo, not 100 Mo.

I compared some dumps at the beginning and end and noticed a lot of objects related to the DataGridView.

(my file was around 1 Mio lines)

The main DataGridView uses VirtualMode so only a few DataGridViewrow should be created (the one displayed + some waiting for GC) but not 25% of the RowCount.

(to be continued)

[fremag]

So I tested with a basic project: one form with one datagridview in virtual mode and 1Mio rows (set with RowCount property). I dumped the memory and noticed only a dozen of DataGridViewRow instances.

So VirtualMode is ok.
I restarted LogExpert with my large file and dumped memory: loading the file tooks a few seconds but there was only a small number of DataGridViewRow.

Then I modified my small project to add this line:

            dataGridView1.CellFormatting += (sender, args) =>
            {
                args.CellStyle.BackColor = dataGridView1.Rows[args.RowIndex].Selected ? Color.Gray : Color.White;
            };

I scrolled up and down, then dumped memory I saw a LOT of DataGridViewRow instances.

I did the same thing with LogExpert: load the big file again and scroll up/down => huge number of DataGridViewRow.

I think the call to dataGridView1.Rows[args.RowIndex].Selected CREATES an instance of DataGridViewRow and is linked to the DataGridView object and never garbage collected.

Loading a big file is not an issue but scrolling is and the tail mode scrolls a lot so the memory used over time increases a lot.

I hope I'm clear and I didn't say any stupid things, any comment is welcome.
I don't have a solution yet for LogExpert.

[hraab]

I remember that there are some conditions in which the DataGridView doesn't dispose the cells/lines even when in virtual mode. For example when lines are selected. It is documented anywhere in the MSDN.

Could be that some code in LogExpert prevents the DataGridView from re-using/disposing lines.

Could you test with highlighting completely disabled and using no columnizer? Also consider to use a debug build and switch off the word highlighting mode (Debug menu). This enables a different cell renderer (older one).

[hraab]

OK, I just read your comment @fremag. This completely makes sense. LogExpert selects the last line when tailing. So I guess this leads to the problem that the DataGridView does not dispose the cells.

[fremag]

Here:
https://github.com/zarunbal/LogExpert/blob/57343b9d5d840fa3bddb298e536ddefb184dc597/src/LogExpert/Controls/LogWindow/LogWindowPrivate.cs#L700

Or here:
https://github.com/zarunbal/LogExpert/blob/6f6cd5f183f18040f1d04e512cb6cc7233363e1c/src/LogExpert/Controls/LogWindow/LogWindowsPublic.cs#L912

[fremag]

Is it possible to call dataGridView.Rows.Clear() sometimes ?

[fremag]

Too late to test this, I need some sleep but tomorrow is another day.
Stay tuned :)

[fremag]

I found two articles about virtual mode:

• virtual mode in datagridview
• Best Practices for Scaling the Windows Forms DataGridView

A row is "shared" (ie one DataGridViewRow object is used to display many virtual rows) until you access to it (Selected property for instance). Then it's cloned and remains in memory (forever ? until you dispose the grid ?)

There is an event to detect when a row is "Unshared":

this.dataGridView.RowUnshared += (sender, args) => Debug.WriteLine("Unshared {0}", args.Row.Index);

The worst is that even a simple call to "grid.Rows[i]" is enough to unshare it...
Look at the DatagridViewCollection's code:

 public DataGridViewRow this[int index] 
        {
            get 
            {
                DataGridViewRow dataGridViewRow = SharedRow(index);
...
                 // unshare row
                  DataGridViewRow newDataGridViewRow = (DataGridViewRow) dataGridViewRow.Clone();
...
                 this.DataGridView.OnRowUnshared(newDataGridViewRow);