stackexchange / stackexchange.dataexplorer

Stack Exchange Data Explorer

Data Explorer

The Stack Exchange Data Explorer is a tool for executing arbitrary SQL queries against data from the various question and answer sites in the Stack Exchange network.

Quick Guide

Prerequisites

• Visual Studio 2015 or later
• .NET Framework 4.7
• SQL Server 2012 Express or later/higher
• IIS7 or later
  • The URL Rewrite 2.0 extension is required for OData endpoint support. If you don't need this, you can comment out the <rewrite> section in web.config.

Layout

• The App directory contains the Data Explorer solution
• The Migrations directory contains the database evolution scripts and a batch file for running them
• The Tools directory contains some precompiled utilities for working with Data Explorer
• The Lib directory contains some 3rd party binaries used in the application
• The SeedData directory contains seed files to populate the Data Explorer schema with some sample data

Configuration

The database can be brought up to date by running the migrate.local.bat file in the Migrations directory. This assumes an existing SQL Server database named DataExplorer with integrated security enabled. If your environment is configured differently, you will need to modify the connection string in your batch file and web.config file to reflect your setup.

Once done, you'll need to populate the Sites table with a record for each site you intend to query against. You can run the sites.sql file in the SeedData directory to prepopulate the table with a small list of popular Stack Exchange sites. To actually run queries, you will need to create additional databases that reflect the connection values in the Sites table; these can optionally be populated with data from the Stack Exchange data dumps using a custom import process or one of the data dump import tools available on Stack Apps.

Contributing

Development

• Install git, or a git client of your choosing (such as TortoiseGit, SourceTree, or GitHub Desktop)
• Fork this repository on GitHub
• Commit changes to your fork, preferably in easy-to-merge branches
• Submit a pull request on GitHub with a description of your changes

Providing Feedback

• Submit bugs, feature requests, and support questions on Meta Stack Exchange or create a new GitHub issue.

Miscellaneous

Third Party Components

• ASP.NET MVC 5
• MiniProfiler
• Dapper
• jQuery
• CodeMirror
• SlickGrid (currently using a fork for updates)
• Flot
• StackExchange.Exceptional
• Json.NET
• DotNetOpenAuth
• reCAPTCHA

For more information, see "Which tools and technologies are used to build Data Explorer?"

stackexchange.dataexplorer's Issues

Outdated readme and google code site, two areas to update

Hi,

The main page: https://github.com/StackExchange/StackExchange.DataExplorer
mentions the old google code procedure for contributing patches.
Also, the google code site should be updated to mention that the project has moved to github.

Cheers,

David

API to execute queries from third party apps

It will be great to have an API so third party tools can execute queries in DataExplorer and use the data.

The same idea than "BigQuery API":https://cloud.google.com/bigquery/docs/reference/v2/ but probably simpler.

Search bar defaults to Stack Overflow on home page

CSV does not include [Post Link] results

If you download the CSV for:

http://data.stackexchange.com/stackoverflow/query/305369/find-posts-in-last-month-with-a-keyword-in-title-body-or-tags

instead of including [Post Link] results for the posts id, it includes the Ids of the posts.

Remember what site you actually ran a query on in history and allow re-running on that site

Used to be, SEDE would always only link to the query for the site you actually last ran it on in history. That was often useful, but there was also a fair use case for just running it on the currently selected site, since usually that's what you'd end up doing anyway.

Recently, SEDE seems to have switched over to favor only the latter case. I'd like to go back to the earlier behavior, or, if possible, add an additional link (parenthesized or otherwise deemphasized?) with the original site. That way the primary and secondary use cases (current and past, respectively) would both be well supported.

Originally reported at Meta SE.

BIT type for Query Parameters

It would be nice to allow the BIT type for query parameters, and have it render as a Checkbox on the client side.

SQL Server 2010 does not exist

https://github.com/StackExchange/StackExchange.DataExplorer

"Prerequisites:
SQL Server 2010 Express or later/higher"

Stabilize SlickGrid sorting

The current implementation of client-side sorting (see issue #19) uses the native JS .sort() method, which is not stable in some browsers (such as, notably, Chrome). This can lead to weird behavior when sorting by a column that contains duplicate values, as shown e.g. in this animation.

There's a couple of possible ways to fix this:

1. Change the SlickGrid data sorting code to use a custom stable sort method, as suggested in this article.
2. Just flat out replace Array.prototype.sort with a stable reimplementation. Here's a suitable drop-in replacement that I've written and added to SOUP.
3. Modify the sort comparator callback to use the current order of the rows as a tie breaker if the compared values are equal. This would be the simplest solution if the previous position of the rows is easily accessible in the comparator, which I'm not sure about.

See also:

• SlickGrid issue #320
• Corresponding report on meta.SE

Periodically changing contents of the "Featured" queries page

Hi everyone,
I believe that the "Featured" tab is handled manually. They are very old queries from 2011-2012. I personally prefer to see different queries from time to time. I think it would be a good idea to change the featured queries periodically based on popularity (for example weekly)