esmero / strawberry_runners Goto Github PK
View Code? Open in Web Editor NEWA post processing Drupal 8/9 module for Strawberryfield dispatched events
License: GNU Lesser General Public License v3.0
A post processing Drupal 8/9 module for Strawberryfield dispatched events
License: GNU Lesser General Public License v3.0
if you save the same ADO multiple times in a row we have no way of knowing/setting that a given processor was enqueued so we keep enqueuing. On process, when e.g 3 run, we generate and store 3 times the derivatives.
The change is not big since we have already a marker inside the Source file but that one gets only updated once one of the queue items is persisted. On pre-persistance we need to check again if a previous one already filled up. If so we discard the processed file, unlink and do nothing
This does not prevent multiple enqueues but ensures that only one gets the proper place in the source ADO.
This things really do not need an ISSUE but since we are more people now (and good people) I will roll with one here.
a dpm()
that was left over In my many tests
Search is disabled when an ADO has no OCR content on it so users are not misled. However, when/if the OCR is included afterwards, the search box doesn't appear until the cache of the entity is cleared.
This is what needs to happen on AbstractPostProcessorQueueWorker::processItem
if ($entity) {
Cache::invalidateTags($entity->getCacheTagsToInvalidate());
}
Comment from @DiegoPino:
Maybe we can do it even later, during the actual Solr Index, because at that level maybe the solr document is not findable yet (because of the tracking/index immediately or not settings)
Relates to esmero/format_strawberryfield#118
MainLoop logic
(see
Enqueue logic
(see
strawberry_runners/test/submit.php
Line 32 in 5c0d688
push item to process on the queue then check mainloop is running or to start
does mainloop need wakeup?
$submitTime = \Drupal::time()->getCurrentTime(); (time when item was enqueued)
set $NxqueuecheckPeriod = 2 * $queuecheckPeriod (safe wait time)
wait (A) mainloop executed one time after submit ($submitTime - $lastRunTime < 0)
OR
wait (B) for $NxqueuecheckPeriod after submit time
If (B) then Mainloop has to be started else if (A) Mainloop already running nothing to do
When I have NLP processing enabled, I'm seeing this error:
bash-5.1# drush queue:run strawberryrunners_process_background --items-limit=1
[error] TypeError: http_build_query(): Argument #1 ($data) must be of type array, null given in http_build_query() (line 67 of /var/www/html/web/modules/contrib/strawberry_runners/src/Web64/Nlp/NlpClient.php) #0 /var/www/html/web/modules/contrib/strawberry_runners/src/Web64/Nlp/NlpClient.php(67): http_build_query(NULL)
#1 /var/www/html/web/modules/contrib/strawberry_runners/src/Web64/Nlp/NlpClient.php(52): Drupal\strawberry_runners\Web64\Nlp\NlpClient->post_call('/status', NULL, 2)
#2 /var/www/html/web/modules/contrib/strawberry_runners/src/Plugin/StrawberryRunnersPostProcessor/OcrPostProcessor.php(362): Drupal\strawberry_runners\Web64\Nlp\NlpClient->get_call('/status', NULL)
#3 /var/www/html/web/modules/contrib/strawberry_runners/src/Plugin/QueueWorker/AbstractPostProcessorQueueWorker.php(612): Drupal\strawberry_runners\Plugin\StrawberryRunnersPostProcessor\OcrPostProcessor->run(Object(stdClass), 1)
The problem seems to begin here, where the $data
argument that is eventually fed into http_build_query here is set to NULL
.
This could be a PHP version thing, where previously http_build_query
would not have complained?
Is this an appropriate fix?:
'content' => is_array($params) && !empty($params) ? http_build_query($params) : "",
We have found that if an ocr process fails, it will throw new RequeueException('I am not done yet. Will re-enqueu myself');
, and continue to retry and fail, creating an infinite loop, as seen in this sample watchdog output:
strawberry_runner_ocr_looping.csv
In our case, I believe the issue is that the command is timing out and may be solved by increasing the ocr processor's timeout setting.
Might the general solution, in part at least, be to switch to using DelayedRequeueException
?
Some system binaries (exiftool
, ffmpeg
) can stream from a remote source without having to generate a local temporary file. The knowledge of which one can/should do this depends on the user to explore but to allow this to happen (e.g fromS3
) we need to generate as argument (instead of %file we can use %url) a signed URL.
This can be a checkbox and allows that processing since e.g Droid
will never be able to read from a remote stream
I think this can be done via SF3S
or using the AWS S3 client
directly, but I need to research it deeper
During background processing of OCR, large images can be generated in temporary storage. We should check if temp files are properly being cleaned up or composted.
Generate a AIPs from ADOs. Do the Atom/Mets thing, bundle, add assets, push to the SWORD api (or any other newer option this people have).
There is legacy code we can look at here:
https://github.com/Islandora-Labs/archidora/blob/7.x/includes/archivematica.inc
We can also build a base API deposit Base Class for these type of plugins so in the future any other final/transformed destination can be built.
@giancarlobi if you have any other deposit/preservation needs let me know. This module is basically your child!
@DiegoPino Probably When a blank page of a PDF with an empty hOCR is feed to Solr doc using ocrhighlight plugin (last release >0.5.0) Solr return an error. As you told me, we cannot present an empty field for miniocr because the XML validation fails.
I can make some checks next days adding here
Sometimes you need to clear queues. Or have a set of objects to reprocess their failed OCR (out of space, random service errors, who know).
We need a VBO action that does exactly this:
Probably the way to go here is:
A small info about "which ones will run/not" as a result at the end of the VBO action
@giancarlobi I just noticed that if someone adds an object, and a PDF is added to queue and then goes and removes the File and changes it for a new one, the queue item gets stuck with an error and keeps adding itself
We may want to make a missing file an action that triggers a queue removal and lease release
e.g this is stuck in the queue right now
{#881 โผ
+"fid": 76
+"nid": "21"
+"asstructure_uniqueid": "urn:uuid:b70b71bf-21f1-49d9-8988-d7b29746ed92"
+"asstructure_key": "as:document"
+"nuuid": "5d6b756b-a246-4105-9fe2-8b706a6384c4"
+"field_name": "field_descriptive_metadata"
+"field_delta": 0
+"metadata": array:14 [โผ
"url" => "s3://525/application-nyhs-v01-n01-quarterly-bulletin-april-1917-1-b70b71bf-21f1-49d9-8988-d7b29746ed92.pdf"
"name" => "NYHS v01 n01, Quarterly Bulletin, April 1917_1.pdf"
"tags" => []
"type" => "Document"
"dr:fid" => 76
"dr:for" => "documents"
"dr:uuid" => "b70b71bf-21f1-49d9-8988-d7b29746ed92"
"checksum" => "525e1209fe0afcae44d8d0a26ea4962a"
"flv:exif" => array:18 [โถ]
"sequence" => 2
"flv:pronom" => array:4 [โถ]
"dr:mimetype" => "application/pdf"
"flv:pdfinfo" => array:28 [โถ]
"crypHashFunc" => "md5"
]
+"force": false
+"plugin_config_entity_id": "pager"
}
This s3://525/application-nyhs-v01-n01-quarterly-bulletin-april-1917-1-b70b71bf-21f1-49d9-8988-d7b29746ed92.pdf file was removed and no longer exists.
Another improvement here:
We should also have an API/smaller kitchen door way of providing Data for a configured @StrawberryRunnersPostProcessor. The idea is that e.g AMI can provide a ready OCR instead of having to go through the whole process. This requires some good planning but I was thinking maybe this:
The Kitchen door will simply deposit for a Given Source File (the UUID) and a given ADO (the ID) a file candidate in a certain location.
Every strawberry_runners_postprocessor instance will check before going full process mode if such exist.
Also: Every strawberry Runner output will also have a file (persisted) representation. This will allow you to build any simple "Text area" element to replace from S3/disk on OCR and trigger Solr reindex.
Matching issue for esmero/strawberryfield#168 and https://github.com/esmero/strawberryfield/issue/165
This will make HOCR processor pass 2 new elements back to the Abstract Processor allowing pure, plain text and an expected total count of documents to be indexed in Solr. The first one is needed for nice Search Excerpts, the second to allow a "harvesting when ready" and saving back into a Frictionless data package at ADO level for long time persistence of generated HOCR. (expensive stuff to generate every time).
@DiegoPino this continue the large post here esmero/format_strawberryfield#11
As part of esmero/strawberryfield#237, ADO Tools need to be removed from this module. The following pieces need to be migrated over:
strawberry_runners.links.task.yml
.strawberry_runners.routing.yml
.UpdateCodeMirrorCommand.php
class.StrawberryRunnersToolsForm.php
class.@DiegoPino I'm thinking that it will be really more simple if we manage flavours one by one per node, that is when we push an item on queue, we load node + flavour, so if we have to manage more than one flavour (i.e. exif and hocr) for the same node we'll load node_id+exif then node_id+hocr.
In addition this help us to set flavours order and to enable single flavour runned ok per node and allow a flavour runs ok also if another flavour runs with errors.
What do you think about this?
Just saw a processor become its own parent after being disabled/dragged/reenabled/dragged. Clearly a bug. Can't be noticed by just looking at that UI list -- with hierarchies-, the issue will become apparent only in the Config YML file (parent: itsownid) and/or when trying to "trigger Post processing" via an action -- bc the Post processor , even if active, will be missing from the list. BUG!
Post processor that creates a zip file for objects comprised of many images.
@DiegoPino & @aksm does this cover the issue? any other details to consider?
As frictionlessdata/datapackage no more needs as sub-dependency illuminate/support, we need a function to replace current str_replace_first used in this module.
Data, singular and plural is great but even better if it can be discovered. How you make that happen? happens that if you push it into Solr It can be discovered!
What type of data? Frictionless Data Packages with a manifest and resources. From CSV to JSON indexes, this will serve tabular, research, Government, publications and Web archiving but also our own OCR which will happily spent its life in a single package longer than in just files around.
Depending on your Docker PHP, Identify might not be able to run on certain JP2s. But EXIF will still (or might) contain width and height, so we can use that to match an attached HOCR document to its processed/informed Size
This came from a friendly talk I had today. For larger deployments the internal Drupal queues may not be scaleable enough, or said differently, we may have macro queues feeding the Drupal Worker driven ones. The need is simple to accomplish. I will add a queue service that can be extended via plugins to feed others/too/instead and then we can figure out if an AWS/rabbit to even a RocksDB one can be added as recipient for pending processes as external services.
Yes, code was hardcoded to deal with only HTTP 1/0 200 OK. Which is like c'mon people! Our newest build of the nlp64
container updated its web server to HTTP 1/1 and now the client is broke.
Solution, extend, override, fix.
Just a small typo here:
and a stray .DS_Store
file in the same folder.
When we produce (from the HOCR/PDFALTO) extraction the pure OCR text we keep the HTML entity encoding. This hurts Views display since internally, twig can not decode the entities and will double encode.
I (just theory) think this can be fixed here
Basically, we don't want this:
Question (if fixing this) is how we remediate/tap into fixing this for existing OCRs. One way would be, on reindex detect if already cached Plain Text has HTML entities, decode and "update" the cache, somewhere here:
https://github.com/esmero/strawberryfield/blob/ce448a0ebe16650df19708459a4600d2c4d2c9e1/src/Plugin/search_api/datasource/StrawberryfieldFlavorDatasource.php#L661 but also could be a hook_update() ?
@aksm what do you think? @alliomeria what do you think? @karomabiles what do you think?
@DiegoPino after our talk, I figure out something like this to annotate flavours into SBF JSON:
* (0) ready, runner executed and STB-JSON updated
* (1) new, runner to execute
* (2) update, runner to execute as STB-JSON was updated
* (3) remove, runner must remove entry related to this flavour
So to mark an image to require "exif" flavour, we write this:
"as:image": {
"urn:uuid:35915592-83c8-40b6-b097-29c51c134cc7": {
"url": "private:\/\/53d\/image-giovane-uomo-del-ballerino-che-indossa-un-salto-russo-piega-del-costume-28730977_3.jpg",
"name": "giovane-uomo-del-ballerino-che-indossa-un-salto-russo-piega-del-costume-28730977_3.jpg",
"tags": [],
"type": "Image",
"dr:fid": 62,
"dr:for": "images",
"dr:uuid": "35915592-83c8-40b6-b097-29c51c134cc7",
"flv:exif": {
"status": 1
},
And after runner update SBF JSON looks like this:
"as:image": {
"urn:uuid:35915592-83c8-40b6-b097-29c51c134cc7": {
"url": "private:\/\/53d\/image-giovane-uomo-del-ballerino-che-indossa-un-salto-russo-piega-del-costume-28730977_3.jpg",
"name": "giovane-uomo-del-ballerino-che-indossa-un-salto-russo-piega-del-costume-28730977_3.jpg",
"tags": [],
"type": "Image",
"dr:fid": 62,
"dr:for": "images",
"dr:uuid": "35915592-83c8-40b6-b097-29c51c134cc7",
"flv:exif": {
"status": 0,
"data": {
"XP Title": "http:\/\/www.dreamstime.com\/royalty-free-stock-photography-young-dancer-man-wearing-folk-russian-costume-jumping-image28730977",
"Copyright": "(c) Stepanov | Dreamstime.com (Photographer) - [None] (Editor)",
"Color Space": "Internal error (unknown value 65535)",
"Exif Version": "Exif Version 2.1",
"X-Resolution": "72",
"Y-Resolution": "72",
"FlashPixVersion": "FlashPix Version 1.0",
"Resolution Unit": "Inch",
"YCbCr Positioning": "Centered"
}
},
Or Strawberryfield data source is totally virtual. During a processing chain we use local storage key values to allow Search API to fetch the recently ingested data. But for a longer/complete reindex we want to have that data in a more stable place, specially for longer running/expensive operations like HORC.
The logic we want is that after a Processor's output has been tracked we push the data into a (new or existing) frictionless data package managed by us file. Idea is if the file exists and the content of a certain Flavor ID is inside we update, if not we create and add.
The the Flavor Data source can always try to fetch from the less expensive Key/Value store if found, or if not, see if the Node itself has one of the packages corresponding to the same FLV id.
Flavors indexed into Solr have the id pattern (Flavor ID)
"ss_search_api_id":"strawberryfield_flavor_datasource/2017:1:en:1d9ae1cd-b3d0-477c-8061-313bb1bc9273:ocr",
Which means:
strawberryfield_flavor_datasource
=> the data source
2017
=> the Node ID
1
= The sequence (remember this is one Node to many files to many sequences)
1d9ae1cd-b3d0-477c-8061-313bb1bc9273
=> The File UUID that was processed
ocr
=> the Plugin type that generated this
Depending on how well I can deal with this issue esmero/strawberryfield#115 we may want to have many Frictionless Data Packages or a single one
The operation would be (pseudo buggy code)
On reindexing/indexing/update from Search API:
@giancarlobi ideas/thoughts?
This is a multi-issue issue. We found a PDF that when processed through PDFALTO did generate correct OCR but also was throwing thousands on PDF standard syntax errors. Because the output of PDFALTO goes to the console directly (terminal) the resulting XML could not be processed. But here is where the larger issue happened, when Hydroponics was set 0 (means run until finishing) the failure was triggering an eternal re-enqueing (I'm pretty sure I coded 3x max retries) and getting stuck for days trying over and over.
-q
as a PDFALTO argument to the OCR processor via the form but this should be a standard argument to be honestFor reference, the command run manually threw this type of syntax errors (PDF Standard non-compliant issue)
Syntax Error (1675049): Incorrect number of arguments in 'sc' command
Syntax Error (1675178): Incorrect number of arguments in 'sc' command
<?xml version="1.0" encoding="UTF-8"?>
<alto xmlns="http://www.loc.gov/standards/alto/ns-v3#" xsi:schemaLocation="http://www.loc.gov/standards/alto/v3/alto.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><Description><MeasurementUnit>pixel</MeasurementUnit>
WACZ now keeps main page and the extra pages in different indexes. take all of them if present
Post processor option to generate HOCR per image page(s), to complement the existing Post processor OCR/HOCR against PDF files.
Per @DiegoPino's notes and discussions:
This will be great to have @DiegoPino! ๐ค
When SBR Runners and/or Hydroponics (So strawberryfield module) are running the messages we generate are the utmost generic and give the Admin user no info of what is happening.
e.g hydroponics will say:
--- processing one item for strawberryrunners_process_background
And internally (really for performance reasons) there is little info coming out from each processor.
Let's make this better.
@alliomeria @karomabiles @aksm let's talk about this when possible.
This is all related to #4 and a call we had with @giancarlobi today (MARCH 17th 2020).
The tasks:
as:
defined file in the JSON.Resuming: This task by itself is just making a particular, slim, and limited version of the generic Binary Processor. Nothing more
I will explain how this needs to be done at the end, when we have all pieces ๐
The idea is that Plugins, which are driven by config entities, as defined here https://github.com/esmero/strawberry_runners/pull/5/files#diff-6cb3b61e72b132f4e76eaf33127a920e are not only sorted by weight but also can/should be hierarchical. Why? Because we would like to allow, by logic, to have Post Processors Plugins to work on other Post processors Outputs. E.g One Post Processor extract files from a PDF, then another uses those files to process HOCR.
Parent
Depth
Which would allow us to use something similar to this form in the Entity List Builder https://api.drupal.org/api/drupal/core%21modules%21system%21tests%21modules%21tabledrag_test%21src%21Form%21TableDragTestForm.php/class/TableDragTestForm/8.7.x
To allow people to move/drag Plugin instances into hierarchy.
Parent
can be NULL (Top Post Processor) or another Post Processor Config Entity UUID/ID
Depth
Can be used to find quickly siblings, etc.
B
can not process until ITEM A
. That is quite complex and we can discuss how to deal with this. Options are (thinking loud)if (strlen(trim($child_node['CONTENT'] ?? "")) > 0) {
Specific Permissions option for accessing ADO Tools
Hiding/not-showing of ADO Tools for Content types not associated with a Strawberryfield (such as Article or Basic Page)
For the new TEXT processor I used a very naive Binary Detection way (mb_detect) which funny enough does not work the same in PHP 7+, 8.0 and 8.1
I'm changing this to a pregmatch
using //u
as detection. This will work!
A post mortem issue that was detected and corrected by @giancarlobi but i ended committing.
808f70c
All attributions go to him but for some reason git history would not accept my co-author command so this is a way of acknowledging that work.
Thanks @giancarlobi. No work should ever go un-accredited!
e.g you have a Book, all is handwritten, the SBR rules say, run the pager and the OCR for all "pages" that have a "tiff", these are tiffs.
We allow this
"ap:tasks": {
"ap:nopost": [
"pager"
]
}
And that will skip the pager for that ADO.
Simple/cool
Remember kids we also have
"ap:tasks": {
"ap:forcepost": true
}
That will force a re-processing even if it was processed before (only if the rules match of course). But ap:nopost
wins. No reprocessing can be forced if we are skipping
Thanks
An OCR bug introduced recently and I'm not getting HOCR working on RC2 for PDFs
Is not accurate for Tesseract generated HOCR (version 4.1.1 +)
A line can look like this
<span class='ocr_line' id='line_1_16' title="bbox 330 1210 706 1234; baseline 0.008 -2; x_size 27.361343; x_descenders 5.3613443; x_ascenders 6">
<span class='ocrx_word' id='word_1_32' title='bbox 330 1212 344 1232; x_wconf 71'>3</span>
With title having 7 or more elements for a single Word.
The fix is simple I just wish I had caught it sooner
@giancarlobi I will fix this asap and also push an update on RC2 docker. This breaks PDF without DJU HOCR
We noticed that we're seeing messages in our logs like this:
"Attempt to compost an unsafe File with path s3://media/cna_001693/cna_001693_metadata.xml was made. Please manually delete it or lower/configure your security settings and code overrides defining what a Safe Path is."
Fortunately, CompostQueueWorker::processItem()
is pretty paranoid about what is safe to delete (thank you Diego!!).
I'm trying to remember how file persistence works in this case where the file has been uploaded to its final destination in S3, and we just provide the path to it in our AMI spreadsheet. Is it just when the source path for the file in the AMI csv is inside the "Relative Path for Persisting Files" that it does not copy the file over and simply saves the path to the file?
My hunch is that AbstractPostProcessorQueueWorker::processItem()
where it sets $needs_localfile_cleanup needs to be aware of whether the file is temporary or not. One way to do that might be to compare the original file location with the saved file location. But I haven't taken the time to figure out where we would find that inside this method.
I'd be very happy to take a whack at a solution for this @DiegoPino if you like and can give me a bit of direction as to the best way to handle this.
See https://ezid.cdlib.org/doc/apidoc.html
I'm not sure yet if this should be an enqueued task but it seems like good idea to maybe make this a post processor and not directly dependent on an Entity Insert/PreSave...
Could work like the WARC to WACZ processor. Under certain rules we request the DOI or ARK ID. If already present then we skip. etc.
While testing for this issue, I found that OCR from some pages was failing to index in solr. As best I can tell/guess, this happens when some OCR with multi-byte characters is serialized and stored in the key_value table, and then fails to unserialize because the character counts in the serialized data don't match the byte counts in that data - or something like that.
It seems weird that something would serialize in a way that can't be unserialized!
Anyway, I'm attaching a couple files. One is a pdf that resulted in the OCR with multibyte characters. The second is the text that I see from the first page in the keystore.
In OcrPostProcessor, where it builds the command to run tesseract, the command always emerges in the form:
{{ ghostscript command that takes the file and tries to generate a png from it }} && {{ tesseract command that uses the png as input }}
I.e. it only works with pdf files as input! Any raster file (tiff, jpeg, etc) results in no OCR being generated.
It should first check the input file to see if tesseract can run on it directly, and if not, then test if ghostscript can convert it into a file that tesseract can run on.
@DiegoPino I'll take this on, it seems pretty easy -- that is unless I have misdiagnosed this problem!
When using the method described here to create a CreativeWorkSeries that is to be displayed using the book viewer with in-book searchable OCR'd text, I'm finding that if I attached more than one image file to one of the page objects, that the book viewer (correctly) displays the extra image file as an extra page, the extra image file is OCR'd and saved and indexed in solr, and the content of that page is searchable in the book viewer. However in-book searching does not find it on the correct page, and the highlight displays on the incorrect page. The smoking gun is, when looking in the key_value table, the name of the ocr record always sets the sequence number as 1.
To answer your question @DiegoPino, the json for the page objects in question do have sequence_id
values, and the file attachments do have correct sequence
values (1, 2, 3 etc.).
I tried re-creating this page object with two image files, rather than first having one image file, saving, and then adding a second image file. No difference - the two images are saved in the key_value table with sequence number of 1.
I spent a while (without xdebug working on my local, alas) trying to isolate where the problem may be happening. The $sequence_number is correct in line 311 of the OcrPostProcessor. However, I noticed here of the AbstractPostProcessorQueueWorker, that it sets the $sequence_key to 1 if $data->siblings is not greater than one. And, I noticed in the strawberryrunners_process_background
queue entries, that while the sequence value is correct, siblings is always set to 1. So that would force the sequence number to always be one.
So maybe this could be fixed by finding where the siblings value is being incorrectly set for multiple-file objects?
We will eventually need to unify word/Doc/HTML/ any document that is uploaded to a single output.
For other things there is Mastercard, but for this there is https://pandoc.org/MANUAL.html!
So happens that Pandadoc is quite powerful and we could add it to the docker container and create also a specific processor. But: i need to build a test set/Matrix to see how the different outputs that we could eventually render in realtime in Archipelago are represented.
@mitchellkeaney (Does Traci has a Github handle? In case i want to include here in the discussion?)
@giancarlobi you ever had the need to uploading a DOCX/PPTX or any other format and have the content indexed in Solr. eg? Any other use cases you have?
During background processing of very large images for OCR, Tesseract stalls/fails, but the processes are lingering and not being killed properly.
Is the timeout being properly calculated? Is something else failing in the kill process?
I waited too long for this so time to make a first branch to complement the good work of @giancarlobi
The idea here is that each post processor is derived from a Plugin. Plugins can/should be able to be connected to each other and have different inputs and outputs. They have settings and can self sustain complex work.
Depending on the number of assets/complexity of the task and where/how something is running they can be enqueued and run sequentially like @giancarlobi demo scripts or could, eventually run via a UI facing batch (need to figure out the exact conditional for this).
The branch associated with this issue will be mostly for now an experiment and should serve as starting ground for generalizing the needed functionality. Goal is to have HOCR as full implementation in the next 14 days. Beta3 will have this working and enabled.
WE have right now processors that feed:
We really just need to make this method
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.