miguelgfierro / codebase

Code base for programming projects

Codebase for programming projects

This is a codebase of useful code pieces in multiple programming languages that I have used in my projects.

To clone:

git clone --recursive [email protected]:miguelgfierro/codebase.git

To update the submodules:

git submodule update --remote --merge

codebase's Issues

Python URL management

https://stackoverflow.com/questions/2018026/what-are-the-differences-between-the-urllib-urllib2-and-requests-module

SQL with python

python pandas value selection/accesing

iter over rows:

df = pd.DataFrame({'a': randn(1000), 'b': randn(1000),'N': randint(100, 1000, (1000)), 'x': 'x'})

%timeit [row.a * 2 for idx, row in df.iterrows()]
# => 10 loops, best of 3: 50.3 ms per loop

%timeit [row[1] * 2 for row in df.itertuples()]
# => 1000 loops, best of 3: 541 µs per loop

plot histograms

Change mains in functions and add docstrings with examples

http://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html

Notation: https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/ensemble/forest.py

Google style guide: https://google.github.io/styleguide/pyguide.html#Comments

flask api html

custom error html
http://flask.pocoo.org/docs/0.12/patterns/errorpages/

plan python structure

Brainstorm:

• - pandas
• - I/O
• - URLs
• - Text transformation
• - Numpy
• - Test management
• - Timer
• - Logger
• - Database
• - Math
• - ML
• - Computer Vision
• - Plot
• - Data generator
• - API

flask api

SQL insert

select top limit ...

SELECT TOP (1) [patient_id]
      ,[number_images]
      ,[array]
  FROM [lung_cancer].[dbo].[scan_images]

image management with opencv

add abstract class in python

SQL linux driver initialization file

apply functions in pandas

http://stackoverflow.com/questions/19798153/difference-between-map-applymap-and-apply-methods-in-pandas

io with opencv

Enable external scripts on SQL

sp_configure
Exec sp_configure 'external scripts enabled', 1
Reconfigure with override

reindex option in random shuffle pandas

http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.reset_index.html

pandas conversion categorical to numeric and viceversa

Take a look at TDSP utilities

https://github.com/Azure/Azure-TDSP-Utilities

image transformation

Resizing
Normalisation
Standardisation
Equalisation

try except when using pkg_resources in case the lib is not found

pkg_resources.DistributionNotFound: The 'cv2' distribution was not found and is required by the application

io with PIL images

create table and drop

python generators

def chunks(l, n):
    """Yield successive n-sized chunks from l."""
    for i in range(0, len(l), n):
        yield l[i:i + n]

Add the output of the shape when ploting with matplot lib in the example

here https://github.com/miguelgfierro/codebase/blob/master/python/plot_base/matplot_base.py

change default location database

Correct docstrings notation removing prints

SQL user&pass generation

plot with opencv

mathematical function: softmax

https://dataaspirant.com/2017/03/02/how-logistic-regression-model-works/

Differential Evolution

Code and test it works

Plot data with matplot lib

Plot image
Add magic to notebook
plot also if it is in grayscale

bokeh examples

https://s3.amazonaws.com/assets.datacamp.com/blog_assets/Python_Bokeh_Cheat_Sheet.pdf

Implement line profiler in the logger

https://github.com/rkern/line_profiler

io

with open(filename) as f:
...


try:
    import cPickle as pickle
except:
    import pickle

io cpp

For CPP: https://github.com/miguelgfierro/hoap3_movement_simulation/blob/master/hoap_navigation/src/HOAP_rave.cpp

see java version

In win and Linux>

java -XshowSettings:properties -version

join several tables

http://stackoverflow.com/questions/9304984/combine-two-tables-in-select-sql-server-2008
http://stackoverflow.com/questions/17582313/correct-way-to-select-from-two-tables-in-sql-server-with-no-common-field-to-join
http://stackoverflow.com/questions/6602229/ms-sql-how-to-select-from-2-tables
http://stackoverflow.com/questions/19247441/how-to-select-2-tables-data-using-single-query-in-mssql

plot array of images

use subplot and accept arrays of images or multidimensional arrays as input.

Add old code from matlab

cpp math

sql order last date

sql_select_newer = """
SELECT TOP(1) *
FROM dbo.test_numpy_io 
WHERE patient_id = ? ORDER BY timet DESC
"""

python sql connector

mongo with python

plot with PIL images

img = Image.open(filename)
plt.imshow(img)
plt.axis('off')
plt.show()

count sql

get number of processors

zip and izip

http://www.saltycrane.com/blog/2008/04/how-to-use-pythons-enumerate-and-zip-to/

flask api json

Implement pandas cheat sheet

https://www.datacamp.com/community/blog/python-pandas-cheat-sheet#gs.e96wwsk