Comments (13)
I'll make the models and we can then meet to discuss
@rcalfredson I have something working now in #36 if you want to schedule a technical discussion with me, @ecumike and yourself.
from page-lab.
First off , we have wanted to build this kind of functionality for some time - so we have had some concepts in mind for a while...
I have looked at the PRs and I think we should discuss your proposal. We have had many ideas on this and I think we can - with very little effort add a UrlFilter
model where we can describe
and name
each filter that you would like to create. This would require some changes to the Url
model as well: where we can break up the url into its constituent parts and do much more efficient queries. We have 7600 urls in the database now, and are expecting the number of urls in our db to exceed 500k, in which case (and perhaps even now), an icontains=
filter is very inefficient and slow.
I imagine there would also be a UrlFilterRule
model that binds the part of the URL you want to filter by to the UrlFilter
. For instance: [hostname
equals
foo.ibm.com
] and [pathname0
equals
marketplace
]
I'm happy to create the models I have in mind here if you like.
Feel free to follow up with a meeting to discuss with myslef and @ecumike
from page-lab.
Thank you for the feedback! Have just seen the message, and think this makes sense. For a chance to meet, there is one meeting that Ann Marie Fred, Tim Figgins, and I are planning to have today at 10:00 AM to talk about filter functionality at large. Do you think that time would be good for us to discuss also? I apologize for the short notice on that, and of course we could meet at a different time if it would work better
Before that time, I will begin to experiment with model-based approach, and hopefully would have something toward that goal to share by then. Thanks again.
from page-lab.
Some of this is internal implementation details, but stepping back from a usability perspective, a simple substring match is the easiest for people to understand.
However, we have a business requirement coming up next (as in this week) to also be able to filter the results by business unit and geography, so keep that in mind, please.
For example:
- Show me all Marketplace pages
- Show me all Marketplace pages in France
- Show me all pages owned by the Analytics business unit
The trick is in figuring out how the framework can allow for that type of filtering, without it getting too specific to our company internals.
from page-lab.
Would like please to share an initial "gist" or sketch of new models based on comment above. Are these similar to the approach you are thinking about?
class UrlFilter(models.Model):
"""
A Url filter to apply.
"""
key = models.CharField(max_length=255, choices=[('host_name', 'host_name'), ('pathname0', 'pathname0')])
class UrlFilterRule(models.Model):
"""
Specific instance of URL filter.
"""
match = models.CharField(max_length=255, null=True)
filter = models.ForeignKey(UrlFilter, on_delete=models.CASCADE)
def getMatches(self):
return [obj for obj in Url.objects.all() if getattr(obj, self.filter.key) == self.match]
Have added an initial one as a test, using some new properties of Url
. My inference is that the changes to Url
would take place on the DB level, rather than on the runtime level via derived properties; do you think this is accurate? Thank you.
from page-lab.
For example, added a temporary property that is pathname0 inside Url
class, defined thus:
@property
def pathname0(self):
return self.url.split('://')[1].split('/')[1]
Please note that this hard-coded way of setting these pathname
properties is only for proof-of-concept (i.e. that it would/could be replaced by some type of iterator).
from page-lab.
Let me create a PR later today / tomorrow to illustrate. I think the filtering data should be persisted
from page-lab.
That sounds good, thank you! Would like please to confirm, it is correct that this would be replacement for, not a companion to, the current proposal? In other words, are there parts of the existing P.R.'s we would want to retain/use, or should they be closed instead?
from page-lab.
These screens should give you an idea on the concept I am working on
There is also a model for the key val pairs in the location.search
, but it is not working yet. PR will be up tomorrow
from page-lab.
Yes, thank you, that sounds good! Have already been working with the code, and the approach is clear to me. Would you want to meet for it today? I have complete availability.
from page-lab.
I wanted to mention that the service and UI of this new functionality should look like this:
a view is created to list all of the UrlFilter
objects at /url-filters/
Each filter corresponds to a url and view that produces the dashboard at the UrlFilter's name: /url-filters/my-url-filter-for-such-and-such
we want to use slugs for the names and make them unique
In the UrlFilter view function we have a very simple operation:
myUrls = UrlFilter.objects.get(name='my-url-filter-name').run_query()
myUrls
are then consumed by the dashboard generating code.
The dashboard generation code may need to be pulled into a helper function
from page-lab.
Closing. In master
as of PR #46
from page-lab.
Agreed, the filters are working. Looking forward to the "AND" and "OR" functionality next!
from page-lab.
Related Issues (20)
- Hide user-timing table if URL has none to show HOT 1
- OCD: Rainy day: Change inline-block to floats HOT 1
- Adjust how average data is displayed for aggregate reports
- Add a way a user can view the test run's Lighthouse report HOT 1
- Change report page chart and data table data set timeframe HOT 3
- Django README edits HOT 1
- Tweak report timeframe to shorter and different intervals and async load it HOT 1
- Merge in #46, fixing conflicts. HOT 1
- It's Database Indexing time!
- Put back global setting context HOT 1
- Custom defined KPI targets
- Bug: Include new URL filter scoping to browse page "load more" button. HOT 1
- Linkable Lighthouse viewer reports HOT 2
- Bug: URL filter - path segment not working as expected HOT 1
- Dashboard charts: Add legend or description of score buckets
- automate Django setup
- automate Node setup
- [RESEARCH PROJECT] Investigate running on Cloud Functions, Lambda, etc
- Cross-Site Scripting: Reflected
- Unused variables
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from page-lab.