nickbruun / hayai Goto Github PK
View Code? Open in Web Editor NEWC++ benchmarking framework
Home Page: https://bruun.co/2012/02/07/easy-cpp-benchmarking
License: Other
C++ benchmarking framework
Home Page: https://bruun.co/2012/02/07/easy-cpp-benchmarking
License: Other
Absolute paths breaks CPack, any project including hayai with add_subdirectory would not be able to use CPack.
Trying with sudo apt-add-repository ppa:bruun/hayai
Yielding error :
Err:5 http://ppa.launchpad.net/bruun/hayai/ubuntu focal Release 404 Not Found [IP: 185.125.190.52 80]
... then further down:
E: The repository 'http://ppa.launchpad.net/bruun/hayai/ubuntu focal Release' does not have a Release file. N: Updating from such a repository can't be done securely, and is therefore disabled by default.
I go through the source codes and can not find the usage about set iterations by command arguments.
Is there a plan for it?
constantine@******:~/hayai-1.0.2$ ./build/sample/sample --help
Usage: sample [OPTIONS]
Runs the benchmarks for this project.
Benchmark selection options:
-l, --list
List the names of all benchmarks instead of running them.
-f, --filter <pattern>
Run only the tests whose name matches one of the positive patterns but
none of the negative patterns. '?' matches any single character; '*'
matches any substring; ':' separates two patterns.
Benchmark execution options:
-s, --shuffle
Randomize benchmark execution order.
Benchmark output options:
-o, --output <format>[:<path>]
Output results in a specific format. If no path is specified, the output
will be presented on stdout. Can be specified multiple times to get output
in different formats. The supported formats are:
console
Standard console output.
json
JSON.
junit
JUnit-compatible XML (very restrictive.)
If multiple output formats are provided without a path, only the last
provided format will be output to stdout.
--c, --color (yes|no)
Enable colored output when available. Default yes.
Miscellaneous options:
-?, -h, --help
Show this help information.
hayai version: 1.0.1
Clock implementation: gettimeofday
Steps to Reproduce:
Expected hayai version: 1.0.2
Actual hayai version: 1.0.1
Very simple piece of example code:
https://bpaste.net/show/7df01dce723f
How can I access benchmark values after running the tests? For example if I needed average iteration times from multiple tests for further analysis?
Would it be possible to add an option to hayai
to save the benchmark results, e.g. in XML
or JSON
format?
It would be nice if you could detect overflows and you are doing your calculations, I ended up having negative value and even very small ones in cases where the test lasts for a while (on low performance hardware).
I suggest starting by using uint64_t instead of signed values, and detect overflows when computing the average times.
I also think you should keep us and s seperated.
As each iteration result is not kept in
hayai/src/hayai_test_result.hpp
Lines 292 to 301 in 9ca0833
ITERATIONS
are only an average of each run. This renders the information rather useless as ITERATIONS
are only a multiple of the RUNS
result.
For example the following benchmark you would expect to see a large variation in each iteration.
#include "hayai.hpp"
int g_sleep = 0;
class SleepFixture : public ::hayai::Fixture {
public:
virtual void SetUp() {
g_sleep += 10000;
std::cout << "SetUp " << g_sleep << std::endl;
usleep(g_sleep);
}
virtual void TearDown() {
g_sleep += 10000;
std::cout << "TearDown " << g_sleep << std::endl;
usleep(g_sleep);
}
void foo() {
g_sleep += 10000;
std::cout << "Sleep " << g_sleep << std::endl;
usleep(g_sleep);
}
};
BENCHMARK_F(SleepFixture, Sleeping, 1, 10) {
foo();
}
Whereas the result shows.
[==========] Running 1 benchmark..
[ RUN ] SleepFixture.Sleeping (1 run, 10 iterations per run)
SetUp 10000
Sleep 20000
Sleep 30000
Sleep 40000
Sleep 50000
Sleep 60000
Sleep 70000
Sleep 80000
Sleep 90000
Sleep 100000
Sleep 110000
TearDown 120000
[ DONE ] SleepFixture.Sleeping (676.609770 ms)
[ RUNS ] Average time: 676609.770 us (~nan us)
Fastest time: 676609.770 us (0.000 us / 0.000 %)
Slowest time: 676609.770 us (0.000 us / 0.000 %)
Median time: 0.000 us (1st quartile: 676609.770 us | 3rd quartile: 676609.770 us)
Average performance: 1.47796 runs/s
Best performance: 1.47796 runs/s (0.00000 runs/s / 0.00000 %)
Worst performance: 1.47796 runs/s (0.00000 runs/s / 0.00000 %)
Median performance: inf runs/s (1st quartile: 1.47796 | 3rd quartile: 1.47796)
[ITERATIONS] Average time: 67660.977 us (~nan us)
Fastest time: 67660.977 us (0.000 us / 0.000 %)
Slowest time: 67660.977 us (0.000 us / 0.000 %)
Median time: 0.000 us (1st quartile: 67660.977 us | 3rd quartile: 67660.977 us)
Average performance: 14.77957 iterations/s
Best performance: 14.77957 iterations/s (0.00000 iterations/s / 0.00000 %)
Worst performance: 14.77957 iterations/s (0.00000 iterations/s / 0.00000 %)
Median performance: inf iterations/s (1st quartile: 14.77957 | 3rd quartile: 14.77957)
[==========] Ran 1 benchmark..
There is no variation in the iterations as this has been averaged away. An example of this can be seen in
hayai/src/hayai_test_result.hpp
Lines 243 to 246 in 9ca0833
Is this a design decision or would it be more ideal to keep the data from each iteration and use this information to collate the minimum, maximum, etc?
I noticed that the json output only lists runs, which I assume includes the setup times. Is is possible to measure the actual benchmark iteration separately? In the console view, is the average iteration time is just (average runtime) / (number of iterations)?
Is there an example of the inclusion filters? I'd love an automatic --hayai_filter=...
commandline option feature, which is parsed by a standard provided hayai::InitHayai(&argc, argv)
function, much like the googletest filters.
Would you be interested in getting rid of the googletest behemot and replace it with header-only https://github.com/philsquared/Catch?
Problem Summary:
Default clock implementation is always gettimeofday because _POSIX_TIMERS
is not found when generating libhayai_main.a
even thought getconf -a
clearly shows that _POSIX_TIMERS is defined and set.
Expected Results:
sample/sample --help
sets clock implementation to clock_gettime(CLOCK_MONOTONIC_RAW)
Actual Results:
sample/sample --help
sets clock implementation to gettimeofday
Environement:
Ubuntu 16.04
getconf -a
ouput:
LINK_MAX 65000
_POSIX_LINK_MAX 65000
MAX_CANON 255
_POSIX_MAX_CANON 255
MAX_INPUT 255
_POSIX_MAX_INPUT 255
NAME_MAX 255
_POSIX_NAME_MAX 255
PATH_MAX 4096
_POSIX_PATH_MAX 4096
PIPE_BUF 4096
_POSIX_PIPE_BUF 4096
SOCK_MAXBUF
_POSIX_ASYNC_IO
_POSIX_CHOWN_RESTRICTED 1
_POSIX_NO_TRUNC 1
_POSIX_PRIO_IO
_POSIX_SYNC_IO
_POSIX_VDISABLE 0
ARG_MAX 2097152
ATEXIT_MAX 2147483647
CHAR_BIT 8
CHAR_MAX 127
CHAR_MIN -128
CHILD_MAX 63645
CLK_TCK 100
INT_MAX 2147483647
INT_MIN -2147483648
IOV_MAX 1024
LOGNAME_MAX 256
LONG_BIT 64
MB_LEN_MAX 16
NGROUPS_MAX 65536
NL_ARGMAX 4096
NL_LANGMAX 2048
NL_MSGMAX 2147483647
NL_NMAX 2147483647
NL_SETMAX 2147483647
NL_TEXTMAX 2147483647
NSS_BUFLEN_GROUP 1024
NSS_BUFLEN_PASSWD 1024
NZERO 20
OPEN_MAX 1024
PAGESIZE 4096
PAGE_SIZE 4096
PASS_MAX 8192
PTHREAD_DESTRUCTOR_ITERATIONS 4
PTHREAD_KEYS_MAX 1024
PTHREAD_STACK_MIN 16384
PTHREAD_THREADS_MAX
SCHAR_MAX 127
SCHAR_MIN -128
SHRT_MAX 32767
SHRT_MIN -32768
SSIZE_MAX 32767
TTY_NAME_MAX 32
TZNAME_MAX 6
UCHAR_MAX 255
UINT_MAX 4294967295
UIO_MAXIOV 1024
ULONG_MAX 18446744073709551615
USHRT_MAX 65535
WORD_BIT 32
_AVPHYS_PAGES 70513
_NPROCESSORS_CONF 8
_NPROCESSORS_ONLN 8
_PHYS_PAGES 4089825
_POSIX_ARG_MAX 2097152
_POSIX_ASYNCHRONOUS_IO 200809
_POSIX_CHILD_MAX 63645
_POSIX_FSYNC 200809
_POSIX_JOB_CONTROL 1
_POSIX_MAPPED_FILES 200809
_POSIX_MEMLOCK 200809
_POSIX_MEMLOCK_RANGE 200809
_POSIX_MEMORY_PROTECTION 200809
_POSIX_MESSAGE_PASSING 200809
_POSIX_NGROUPS_MAX 65536
_POSIX_OPEN_MAX 1024
_POSIX_PII
_POSIX_PII_INTERNET
_POSIX_PII_INTERNET_DGRAM
_POSIX_PII_INTERNET_STREAM
_POSIX_PII_OSI
_POSIX_PII_OSI_CLTS
_POSIX_PII_OSI_COTS
_POSIX_PII_OSI_M
_POSIX_PII_SOCKET
_POSIX_PII_XTI
_POSIX_POLL
_POSIX_PRIORITIZED_IO 200809
_POSIX_PRIORITY_SCHEDULING 200809
_POSIX_REALTIME_SIGNALS 200809
_POSIX_SAVED_IDS 1
_POSIX_SELECT
_POSIX_SEMAPHORES 200809
_POSIX_SHARED_MEMORY_OBJECTS 200809
_POSIX_SSIZE_MAX 32767
_POSIX_STREAM_MAX 16
_POSIX_SYNCHRONIZED_IO 200809
_POSIX_THREADS 200809
_POSIX_THREAD_ATTR_STACKADDR 200809
_POSIX_THREAD_ATTR_STACKSIZE 200809
_POSIX_THREAD_PRIORITY_SCHEDULING 200809
_POSIX_THREAD_PRIO_INHERIT 200809
_POSIX_THREAD_PRIO_PROTECT 200809
_POSIX_THREAD_ROBUST_PRIO_INHERIT
_POSIX_THREAD_ROBUST_PRIO_PROTECT
_POSIX_THREAD_PROCESS_SHARED 200809
_POSIX_THREAD_SAFE_FUNCTIONS 200809
_POSIX_TIMERS 200809
TIMER_MAX
_POSIX_TZNAME_MAX 6
_POSIX_VERSION 200809
_T_IOV_MAX
_XOPEN_CRYPT 1
_XOPEN_ENH_I18N 1
_XOPEN_LEGACY 1
_XOPEN_REALTIME 1
_XOPEN_REALTIME_THREADS 1
_XOPEN_SHM 1
_XOPEN_UNIX 1
_XOPEN_VERSION 700
_XOPEN_XCU_VERSION 4
_XOPEN_XPG2 1
_XOPEN_XPG3 1
_XOPEN_XPG4 1
BC_BASE_MAX 99
BC_DIM_MAX 2048
BC_SCALE_MAX 99
BC_STRING_MAX 1000
CHARCLASS_NAME_MAX 2048
COLL_WEIGHTS_MAX 255
EQUIV_CLASS_MAX
EXPR_NEST_MAX 32
LINE_MAX 2048
POSIX2_BC_BASE_MAX 99
POSIX2_BC_DIM_MAX 2048
POSIX2_BC_SCALE_MAX 99
POSIX2_BC_STRING_MAX 1000
POSIX2_CHAR_TERM 200809
POSIX2_COLL_WEIGHTS_MAX 255
POSIX2_C_BIND 200809
POSIX2_C_DEV 200809
POSIX2_C_VERSION 200809
POSIX2_EXPR_NEST_MAX 32
POSIX2_FORT_DEV
POSIX2_FORT_RUN
_POSIX2_LINE_MAX 2048
POSIX2_LINE_MAX 2048
POSIX2_LOCALEDEF 200809
POSIX2_RE_DUP_MAX 32767
POSIX2_SW_DEV 200809
POSIX2_UPE
POSIX2_VERSION 200809
RE_DUP_MAX 32767
PATH /bin:/usr/bin
CS_PATH /bin:/usr/bin
LFS_CFLAGS
LFS_LDFLAGS
LFS_LIBS
LFS_LINTFLAGS
LFS64_CFLAGS -D_LARGEFILE64_SOURCE
LFS64_LDFLAGS
LFS64_LIBS
LFS64_LINTFLAGS -D_LARGEFILE64_SOURCE
_XBS5_WIDTH_RESTRICTED_ENVS XBS5_LP64_OFF64
XBS5_WIDTH_RESTRICTED_ENVS XBS5_LP64_OFF64
_XBS5_ILP32_OFF32
XBS5_ILP32_OFF32_CFLAGS
XBS5_ILP32_OFF32_LDFLAGS
XBS5_ILP32_OFF32_LIBS
XBS5_ILP32_OFF32_LINTFLAGS
_XBS5_ILP32_OFFBIG
XBS5_ILP32_OFFBIG_CFLAGS
XBS5_ILP32_OFFBIG_LDFLAGS
XBS5_ILP32_OFFBIG_LIBS
XBS5_ILP32_OFFBIG_LINTFLAGS
_XBS5_LP64_OFF64 1
XBS5_LP64_OFF64_CFLAGS -m64
XBS5_LP64_OFF64_LDFLAGS -m64
XBS5_LP64_OFF64_LIBS
XBS5_LP64_OFF64_LINTFLAGS
_XBS5_LPBIG_OFFBIG
XBS5_LPBIG_OFFBIG_CFLAGS
XBS5_LPBIG_OFFBIG_LDFLAGS
XBS5_LPBIG_OFFBIG_LIBS
XBS5_LPBIG_OFFBIG_LINTFLAGS
_POSIX_V6_ILP32_OFF32
POSIX_V6_ILP32_OFF32_CFLAGS
POSIX_V6_ILP32_OFF32_LDFLAGS
POSIX_V6_ILP32_OFF32_LIBS
POSIX_V6_ILP32_OFF32_LINTFLAGS
_POSIX_V6_WIDTH_RESTRICTED_ENVS POSIX_V6_LP64_OFF64
POSIX_V6_WIDTH_RESTRICTED_ENVS POSIX_V6_LP64_OFF64
_POSIX_V6_ILP32_OFFBIG
POSIX_V6_ILP32_OFFBIG_CFLAGS
POSIX_V6_ILP32_OFFBIG_LDFLAGS
POSIX_V6_ILP32_OFFBIG_LIBS
POSIX_V6_ILP32_OFFBIG_LINTFLAGS
_POSIX_V6_LP64_OFF64 1
POSIX_V6_LP64_OFF64_CFLAGS -m64
POSIX_V6_LP64_OFF64_LDFLAGS -m64
POSIX_V6_LP64_OFF64_LIBS
POSIX_V6_LP64_OFF64_LINTFLAGS
_POSIX_V6_LPBIG_OFFBIG
POSIX_V6_LPBIG_OFFBIG_CFLAGS
POSIX_V6_LPBIG_OFFBIG_LDFLAGS
POSIX_V6_LPBIG_OFFBIG_LIBS
POSIX_V6_LPBIG_OFFBIG_LINTFLAGS
_POSIX_V7_ILP32_OFF32
POSIX_V7_ILP32_OFF32_CFLAGS
POSIX_V7_ILP32_OFF32_LDFLAGS
POSIX_V7_ILP32_OFF32_LIBS
POSIX_V7_ILP32_OFF32_LINTFLAGS
_POSIX_V7_WIDTH_RESTRICTED_ENVS POSIX_V7_LP64_OFF64
POSIX_V7_WIDTH_RESTRICTED_ENVS POSIX_V7_LP64_OFF64
_POSIX_V7_ILP32_OFFBIG
POSIX_V7_ILP32_OFFBIG_CFLAGS
POSIX_V7_ILP32_OFFBIG_LDFLAGS
POSIX_V7_ILP32_OFFBIG_LIBS
POSIX_V7_ILP32_OFFBIG_LINTFLAGS
_POSIX_V7_LP64_OFF64 1
POSIX_V7_LP64_OFF64_CFLAGS -m64
POSIX_V7_LP64_OFF64_LDFLAGS -m64
POSIX_V7_LP64_OFF64_LIBS
POSIX_V7_LP64_OFF64_LINTFLAGS
_POSIX_V7_LPBIG_OFFBIG
POSIX_V7_LPBIG_OFFBIG_CFLAGS
POSIX_V7_LPBIG_OFFBIG_LDFLAGS
POSIX_V7_LPBIG_OFFBIG_LIBS
POSIX_V7_LPBIG_OFFBIG_LINTFLAGS
_POSIX_ADVISORY_INFO 200809
_POSIX_BARRIERS 200809
_POSIX_BASE
_POSIX_C_LANG_SUPPORT
_POSIX_C_LANG_SUPPORT_R
_POSIX_CLOCK_SELECTION 200809
_POSIX_CPUTIME 200809
_POSIX_THREAD_CPUTIME 200809
_POSIX_DEVICE_SPECIFIC
_POSIX_DEVICE_SPECIFIC_R
_POSIX_FD_MGMT
_POSIX_FIFO
_POSIX_PIPE
_POSIX_FILE_ATTRIBUTES
_POSIX_FILE_LOCKING
_POSIX_FILE_SYSTEM
_POSIX_MONOTONIC_CLOCK 200809
_POSIX_MULTI_PROCESS
_POSIX_SINGLE_PROCESS
_POSIX_NETWORKING
_POSIX_READER_WRITER_LOCKS 200809
_POSIX_SPIN_LOCKS 200809
_POSIX_REGEXP 1
_REGEX_VERSION
_POSIX_SHELL 1
_POSIX_SIGNALS
_POSIX_SPAWN 200809
_POSIX_SPORADIC_SERVER
_POSIX_THREAD_SPORADIC_SERVER
_POSIX_SYSTEM_DATABASE
_POSIX_SYSTEM_DATABASE_R
_POSIX_TIMEOUTS 200809
_POSIX_TYPED_MEMORY_OBJECTS
_POSIX_USER_GROUPS
_POSIX_USER_GROUPS_R
POSIX2_PBS
POSIX2_PBS_ACCOUNTING
POSIX2_PBS_LOCATE
POSIX2_PBS_TRACK
POSIX2_PBS_MESSAGE
SYMLOOP_MAX
STREAM_MAX 16
AIO_LISTIO_MAX
AIO_MAX
AIO_PRIO_DELTA_MAX 20
DELAYTIMER_MAX 2147483647
HOST_NAME_MAX 64
LOGIN_NAME_MAX 256
MQ_OPEN_MAX
MQ_PRIO_MAX 32768
_POSIX_DEVICE_IO
_POSIX_TRACE
_POSIX_TRACE_EVENT_FILTER
_POSIX_TRACE_INHERIT
_POSIX_TRACE_LOG
RTSIG_MAX 32
SEM_NSEMS_MAX
SEM_VALUE_MAX 2147483647
SIGQUEUE_MAX 63645
FILESIZEBITS 64
POSIX_ALLOC_SIZE_MIN 4096
POSIX_REC_INCR_XFER_SIZE
POSIX_REC_MAX_XFER_SIZE
POSIX_REC_MIN_XFER_SIZE 4096
POSIX_REC_XFER_ALIGN 4096
SYMLINK_MAX
GNU_LIBC_VERSION glibc 2.23
GNU_LIBPTHREAD_VERSION NPTL 2.23
POSIX2_SYMLINKS 1
LEVEL1_ICACHE_SIZE 32768
LEVEL1_ICACHE_ASSOC 8
LEVEL1_ICACHE_LINESIZE 64
LEVEL1_DCACHE_SIZE 32768
LEVEL1_DCACHE_ASSOC 8
LEVEL1_DCACHE_LINESIZE 64
LEVEL2_CACHE_SIZE 262144
LEVEL2_CACHE_ASSOC 8
LEVEL2_CACHE_LINESIZE 64
LEVEL3_CACHE_SIZE 8388608
LEVEL3_CACHE_ASSOC 16
LEVEL3_CACHE_LINESIZE 64
LEVEL4_CACHE_SIZE 0
LEVEL4_CACHE_ASSOC 0
LEVEL4_CACHE_LINESIZE 0
IPV6 200809
RAW_SOCKETS 200809
_POSIX_IPV6 200809
_POSIX_RAW_SOCKETS 200809
Exclude noise from measured time.
I'd like to measure time of some function. This function take variable of user defined type and modify it. So for repeated calling of this function I need restore input parameters for each iteration. As result, the measured time of benchmark it time which was taken for execution of function (useful) and for restoring input parameter (noise).
Is there way to exclude this noise from report?
I've refactored hayai's build instructions and reorganized the files a bit, but haven't changed any actual code. This was to support making your library available to the ROS build system (www.ros.org). How would you like the license to be handled? I'm thinking just leave it as is, but add a new line for a URL pointing to my fork? I don't expect you to pull these changes back in. I'd appreciate some feedback/guidance on how you'd like me to proceed.
I was trying to create a fixture so that the size of the input of the test would be different among benchmarks, but with the same fixture.
Hawai doesn't do well with template fixtures, I expected as much. However I think it would be interesting to give SetUp, TearDown etc a void* parameter so that we can configure those steps per benchmark.
When using hayai
as a submodule, I think we should be able to disable the installation of the library and the compilation of the sample
target. I've never contributed to anything on github but I can take care of it myself if you want.
Adding simple options would do.
warning C4244: 'return' : conversion from 'const uint64_t' to 'double', possible loss of data hayai_test_result.hpp line 52
warning C4244: 'return' : conversion from 'const uint64_t' to 'double', possible loss of data hayai_test_result.hpp line 73
warning C4244: 'return' : conversion from 'const uint64_t' to 'double', possible loss of data hayai_test_result.hpp line 80
Hi,
It seems there's not a way to stop/fail a benchmark whenever it is not possible to continue during SetUp().
class TestFixture : public ::hayai::Fixture
{
public:
virtual void SetUp() {
test = new Test();
if (!test->canContinue()) {
// SHOULD STOP HERE!
}
}
virtual void TearDown() {
delete(test);
}
Test *test;
};
BENCHMARK_F()'s connected to the fixture above still continues to execute.
Cheers!
The only way I was able to do it is by hacking and removing const
here:
hayai/src/hayai_benchmarker.hpp
Line 287 in 9bdca1c
Hi,
this is a very interesting project. I would like to try and use at work. However, personal licenses are not compatible with our work policies. Would you be willing to change the license to something well established? For example one of these? The Apache License 2.0 seems to be what you are aiming for... Unfortunately without the beer ๐
Thanks for considering!
Hi,
I have been looking over the framework - but not actually tried any thing with it yet. Hence, if the below does not make sense, then you may close the issue.
From inspection it looks like the TestBody() timing includes overhead for both calls to the clock and the loop consideration.
If I look at the code in hayai_test.hpp:49:
startTime = Clock::Now();
// Run the test body for each iteration.
std::size_t iteration = iterations;
while (iteration--)
TestBody();
// Get the ending time.
endTime = Clock::Now();
endTime - startTime would also represent the time taken to return from the functions Clock::Now() , processing of iteration--
and the function call to TestBody
.
Would it not be better to:
--iteration
(prefix is generally faster), but also bench the time taken (probably very small, so might be too negligible) to do the subtraction and the branch consideration (i.e. the while
condition)TestBody()
that can negate the call time over a virtual function callIn my mind, currently:
Test Time = Time in body + Time in framework
While Time in framework
may stay relatively static, there is no way to know how much time is in the framework so exact timings of the TestBody()
call is not known.
If you take consideration of the 1,2 and 3 points, the time can be removed from the final calculation to formulate a more accurate timing.
Would this be worthwhile?
Regards,
Richard
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.