Memory leak when using Vizceral's updateData function about vizceral

[gupta1]

Can someone please look into it? Is there any way i can do a quick fix in my code to avoid it?

[jrsquared]

There may be a small memory leak, but we consistently run this for hours and don't see anything substantial. You will see Float32Array, BufferAttribute and WebGlBuffers grow until the garbage collector comes and collects them. The only thing that I can think of is if you are actually adding new nodes/connections every time you call updateData. updateData culls all old nodes and connections.

[gupta1]

Thanks for your reply Justin!

According to my observations, if the browser tab remains active, then the heap size eventually keeps increasing. I have seen it grow from 90mb to 500mb. In your scenario, is the increase comparative to this?

I didn't quite understand what you meant by last 2 lines. Everytime I refresh, I am creating a new json with connections and nodes(these are mostly same nodes as before but can be different too depending on refresh time or if user changed main query which completely changes response). Is that approach not correct? Currently, i get a response which I convert to vizceral understandable format and then feed it to updateData() on refresh.

I will paste relevant code, please let me know if something is missing or if I am doing in incorrect way. Here is my code snippet:

        $scope.createVizceralJson = function (response) {
            var vizceralBuckets = $scope.vis.aggs['buckets'];

            var item = {};
            item["renderer"] = "global";
            item["name"] = "edge";
            item["entryNode"] = "Master";
            var nodes = [{
                "renderer": "region",
                "name": "Master",
                "class": "normal"
            }];

            item["nodes"] = $scope.createNodes(response, nodes, vizceralBuckets, 0);              // Duplicate nodes
            item["connections"] = $scope.createConnections(response, true, vizceralBuckets);

            return item;
        }

        function configureVizceral(viz) {
            if ($scope.vis.params.darkTheme) {
                viz.updateStyles($laConstants.Vizceral.darkThemeColor);
            } else {
                viz.updateStyles($laConstants.Vizceral.lightThemeColor);
            }
            var label = $scope.vis.params.customLabel ? $scope.vis.params.customLabel : 'RPS';
            var bottomLabel = $scope.vis.params.customBottomLabel ? $scope.vis.params.customBottomLabel : 'ERROR RATE';
            var definitions = {
                detailedNode: {
                    volume: {
                        default: {
                            top: { header: label, data: 'data.volumePercent', format: '0.0%' },
                            bottom: { header: bottomLabel, data: 'data.classPercents.danger', format: '0.00%' },
                            donut: {},
                            arc: {}
                        },
                        focused: {
                            top: { header: label, data: 'data.volume', format: '0,0' },
                            bottom: { header: bottomLabel, data: 'data.classPercents.danger', format: '0.00%' },
                        },
                        entry: {
                            top: { header: 'TOTAL ' + label, data: 'data.volume', format: '0,0' },
                            bottom: { header: bottomLabel, data: 'data.classPercents.danger', format: '0.00%' },
                        }
                    }
                }
            }

            viz.updateDefinitions(definitions);
        }

        function updateVizceral(viz, resp) {
            var responseJson = $scope.createVizceralJson(resp.Response.Tables[0]);
            viz.updateData(responseJson);     
        }

        $scope.$watch('esResponse', function (resp) {
            if (resp) {
                if (!viz) {
		    var canvas = "<canvas id='vizceral'></canvas>";
		    angular.element('#vizceral-parent').append(canvas);
		    viz = new Vizceral.default(document.getElementById('vizceral'));
                    configureVizceral(viz);
                    updateVizceral(viz, resp);
                    viz.setView();
                    viz.animate();
                }
                else {
                    configureVizceral(viz);                    
                    updateVizceral(viz, resp);
                }
                viz.setOptions({ 'allowDraggingOfNodes': $scope.vis.params.allowDraggingOfNodes, 'showLabels': $scope.vis.params.showLabels });
            }
        });

Thanks a lot for your help everyone!

[jrsquared]

By the last two lines I meant creating new, unique nodes on every update, with changing names every time.

Based on your example nothing is jumping out at me as causing a memory leak, but I do see that you are calling setOptions, updateStyles and updateDefinitions every time you get new data as well. I haven't tested if there's a memory leak in there, but maybe calling all of those only once in the setup would be a good start?

[gupta1]

Thanks for explaining Justin!

I tried it only with updateData() function. I changed the data to the sample_data.json given in vizceral_example and now I am calling refresh on it and feeding the same json again and again. Still I see memory leak. I have seen it going from 99.00 mb to 416mb. I am guessing it will eventually crash. Currently the refresh time is 5 seconds, so I am calling updateData() every 5 seconds on the same sample json.
Is there any suggestion for quick fix which I can use?

Here is the code I am testing as of now:

        function updateVizceral(viz, resp) {
            viz.updateData({sample_data.json});    // Writing it as sample_data.json because it is huge.
        }

        $scope.$watch('esResponse', function (resp) {
            if (resp) {
                if (!viz) {
                    var canvas = "<canvas id='vizceral'></canvas>";
		    angular.element('#vizceral-parent').append(canvas);
		    viz = new Vizceral.default(document.getElementById('vizceral'));
                    updateVizceral(viz, resp);
                    viz.setView();
                    viz.animate();
                }
                else {                
                    updateVizceral(viz, resp);
                }
            }
        });

Adding the snapshot of memory heap comparison between first and last heap size:

[gupta1]

@jrsquared Can you please take a look at this and provide any suggestions that you might have for further investigation?

Thanks for your help!

[jrsquared]

I honestly don't know why this would be happening; I would have to dig in with your code and graph data.

If you could provide a working code example that exhibits the doesn't have angular and with a data source that causes the memory leak, I would be happy to dig in.

[jrsquared]

Closing since it has been over a month without any response. Feel free to reopen if it's still happening and you can provide a working code sample that exhibits this behavior.