tomudding / 2ioa0-dbl-hti-webtech Goto Github PK

View Code? Open in Web Editor NEW

2.0 2.0 2.0 96.7 MB

JavaScript 0.01% HTML 0.97% CSS 0.05% Python 0.09% Jupyter Notebook 98.78% Java 0.01% PostScript 0.01% TeX 0.08%

2ioa0-dbl-hti-webtech's People

Contributors

Stargazers

Watchers

Forkers

baiguo0016 vaols

2ioa0-dbl-hti-webtech's Issues

Implement the node-link hierarchical layout[Sophia]

3.3 Improve Running time of nodelink [Sophia & Sam] [1 - 4 hrs]

5.6 Implementing loading screen for the adjacency matrix [Sam] [2Hrs]

4.2 Implement the design of the web app prototype [Anton] [5hr]

6.1 Write initial script video and especially make the intro part ready[Tim] [3hr]

3.2.2 Implementing node-link diagram [Sam] [6hr]

5.2 Converge the new design with the old structure of the webpage [Anton, Tom] [3 - 4] hrs

4.4 Add the original adjacency matrix [Yuqin] [2hrs]

8.5 Fix no connection from Nginx to Bokeh on the server. [Tom] [8hr]

4.3 Actually reorder the original pandas dataframe instead of displaying the reordered distance matrix [Yuqin] [2hr]

3.2 Fix the hierarchical and force-directed NL [Tom] [2 Hr]

3.2.1 Researching how to display node-link diagram from pandas dataframe [Sam] [2hr]

2.1 Make a single script for converting csv to panda df + cleaning [Sam & Steven] [2hr]

5.5 Implement Loadingscreen Prototype [Sam] [1Hr]

3.1 Add more features/interactions to nodelink [Sophia & Sam] [6-10 hrs]

3.2 Read node-link layout literature [Sam, Sophia] [2hr per person]

5.7 Make the website responsive [Anton] [2-3] hrs

4.1 Read matrix reordering literature [Steven, Yuqin Cui] [4hr]

1.2 Enable Flask to run on the web server [Tom] [2hr]

Flask through gunicorn and nginx

1.3 Establish proper modularisation of the Flask app and add documentation for the rest of the team [Tom] [4hr]

4.2 Implementing reorderings [Steven] [Yuqin Cui] [5hr]

3.5 Implement hierarchical nodelink layout. [Sophia]

8.1 Create/improve bokeh implementation within Flask [Tom] [6hr]

5.1 Discuss the interface of the tool in the visualization page. With 5.10 [everyone] [1 Hr] [FRIDAY]

2.2 & 3.6 Slider selection weight python lambda for node-link. [Sophia][3hrs]

5.3 Getting inspiration for Loading Screen [Sam] [1Hr]

1.4 Create data upload system [Tom] [4hr]

8.4 Improve Bokeh server & application implementation with Flask [Steven & Tom] [6hr]

5.8 Re-implement the loading screens, or replace them by one loading screen [Tim] [1 Hr]

1.1 Configure server for outside access [Tom] [2hr]

Configure the web server to be able to communicate with the outside world.

3.5 & 4.3 Reading Interaction Literature [Sam] [0.5 Hr]

5.4 Look into D3 documentation for the loading screen [Sam] [2Hrs]

4.1 Design a web app outlook for the first prototype. [Anton, Sophia(Yuqing Zeng)] [1hr]

Removal of 'website' folder in master branch

@Yvee1 can the folder website (and its contents) be removed from master? Since it was used to test sigma.js, I feel like it does no longer have a purpose to be there.

1.1 / 5.2 / 8.2 Implement file showcase after uploading a file [Tom] [2hr]

5.10 What interactions&functions do we need to have. [to be discussed on Friday meeting]

3.1 Display in Adjacency Matrix [Steven, Yuqin Cui] [6hr]

10 Fill out peer review forms [everyone] [0.5 Hrs] [FRIDAY]

WebGL does not correctly render on macOS

Currently we use force bokeh to use WebGL to speed up the rendering process. However, this does not correctly work in Safari (tested on macOS 10.14.4) browsers.

8.1 Implement proper error handling on the back-end part of graphion [Tom] [2hr]

7.1 Write the node-link part of the report. [Sam] [2 Hrs]

8.1 Write Report Abstract , Introduction[Tim] [3hr]

0.1 Installing the development environment [Everyone] [2hr]

6.1 Finalize the video script, distribute the parts to everyone. [everyone] [1 hr] [FRIDAY]

8.2 Separating graphing functions and improving running time (see 3.3) [Tom] [4hr]

5.1 Improve the design of the website [Anton, Sophia(4hrs)]

9.1 Implement 3D node-link visualization [Steven] [12 hrs]

7.1 Reading and checking the report [Everyone] [1hr]

Speed issues with HV -> Bokeh -> HTML

@TechWizzart I partially updated the code you wrote for the radial node link diagram as can be seen in my feature-directedradianodelinkgraph branch:

2IOA0-DBL-HTI-Webtech/test.py

Lines 11 to 12 in 8175f10

    
           with open("datasets/GephiMatrix_co-citation.csv", 'r') as csvfile: 
        
               dialect = csv.Sniffer().sniff(csvfile.readline())

2IOA0-DBL-HTI-Webtech/test.py

Lines 18 to 30 in 8175f10

    
           df = pd.DataFrame() 
        
           for chunk in pd.read_csv("datasets/GephiMatrix_co-citation.csv", sep=dialect.delimiter, mangle_dupe_cols=True, index_col=False, chunksize=1000): 
        
               df = pd.concat([df, chunk], ignore_index=True) 
        
           #df.index.name = "delete" 
        
           #df = df.drop(index='delete') 
        
           nodes = df.columns.values.tolist() 
        
           nodes.pop(0) 
        
           #print(nodes) 
        
           df["Unnamed: 0"] = nodes 
        
           df = df.rename(columns={'Unnamed: 0': 'name'}) 
        
           df = df.set_index(keys='name')

I don't know which code is faster, but this code is better for larger datasets. However, I have another problem now, The whole matrix can be imported into networkx is in ~1s (this includes the the whole content fetching part of the process). However, it takes bokeh 60s to generate the HTML for the plot and that is when I datashade and bundle the graph.

I need your input on speeding up the part which exports it to HTML because I have no clue how to do it.

4.1 Brushing and linking of different views [Sam & Steven][6 Hrs]

Cannot be done before we have properly designed the interface.

	with open("datasets/GephiMatrix_co-citation.csv", 'r') as csvfile:
	dialect = csv.Sniffer().sniff(csvfile.readline())

	df = pd.DataFrame()
	for chunk in pd.read_csv("datasets/GephiMatrix_co-citation.csv", sep=dialect.delimiter, mangle_dupe_cols=True, index_col=False, chunksize=1000):
	df = pd.concat([df, chunk], ignore_index=True)

	#df.index.name = "delete"
	#df = df.drop(index='delete')

	nodes = df.columns.values.tolist()
	nodes.pop(0)
	#print(nodes)
	df["Unnamed: 0"] = nodes
	df = df.rename(columns={'Unnamed: 0': 'name'})
	df = df.set_index(keys='name')