createamind / busyplan Goto Github PK
View Code? Open in Web Editor NEWplan
plan
简单的测试环境
提供传感器数据接口,将原始数据处理成可用的数据类型发送给感知决策模块
控制接口,接收决策模块的控制指令,方向盘转角、油门、刹车,将处理后的控制指令发送给执行器
gazebo,torcs,真车
目的:??(不知道和公司发展有什么关系,也不知道做出来能干什么……)
zdx:序列预测是智能非常重要的能力,对于AI非常重要,完全符合公司目标通用智能,做出了能增强现有神经网络的智能。
具体场景:大家一起想!避障,其他车辆意图的预测,torcs游戏验证?机器人自己动作的预测,常识学习。
原型验证ok,完善中再继续找应用的场景和产品的具体完善。
目标:搭建一个视频生成网络
要求:pix2pix 框架,基于GAN技术
注:以上为主观因素
基本结构:
G:简易 3D_UNET 网络,初步大小64 x 64,目标大小 128 x 128
D:C3D 类似结构判别器
效果: 输入10帧视频,输出5帧视频
预计时间:总用时 8 周
15 号最新更新:
按张总的意思,换 Pytorch 框架,基于 pix2pix 原始代码修改修改。参考如下:
pix2pix pytorch 源代码:
https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix
一些 pytorch 3D 应用的实例:
https://github.com/shiba24/3d-unet
https://github.com/kenshohara/video-classification-3d-cnn-pytorch
https://github.com/kenshohara/3D-ResNets-PyTorch
pytorch 官方 Document:
http://pytorch.org/docs/master/nn.html
关键 operatoin:
3D deconvolution - torch.nn.ConvTranspose3d
3D convolution - torch.nn.Conv3d
3D maxpooling - torch.nn.MaxPool3d
3D dropout - torch.nn.Dropout3d
Keras 实现 已取消
计划注意:
可能失败原因
1.论文 https://arxiv.org/abs/1611.06624
代码 https://github.com/pfnet-research/tgan https://github.com/dandelin/Temporal-GAN-Pytorch
2.论文 http://carlvondrick.com/tinyvideo/paper.pdf
网站 http://web.mit.edu/vondrick/tinyvideo/
代码https://github.com/cvondrick/videogan
gan video 两周
第一周, 熟悉已有论文,选用一个算法复现。使用自己的数据集,调参。
第二周,分析隐变量的语义相关信息,自动驾驶的转向角度和z的关系 。
http://cs231n.stanford.edu/reports/2017/pdfs/200.pdf
1.multi-scale deep network, outperformed most other meth- ods in nearly every metric. Inspection of the output maps, however, shows that the images produced are extremely blurry. So while they are able to achieve low average er- ror, their utility for practical depth mapping applications is limited.
生成的深度图模糊,原因在于优化目标是平均像素误差。
2.CycleGAN is able to best retain the image features with clear definition, but often with high error in the depth-space representation.
生成深度图比较清晰,特征重建较好,而像素级误差较大,原因在于优化目标是特征级误差。
3.改进方向
设计损失函数,使其能同时优化像素级误差和特征级误差。
目的:验证现有模型预测深度的可靠性,为是否进一步改良模型提供依据。
2.- [ ] Deeper Depth Prediction with Fully Convolutional Residual Networks upsampling to 640x480
https://arxiv.org/pdf/1606.00373.pdf
https://github.com/iro-cp/FCRN-DepthPrediction 代码不完整
4.Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields https://arxiv.org/pdf/1502.07411.pdf
6.Single-Image Depth Perception in the Wild
https://arxiv.org/pdf/1604.03901.pdf
https://github.com/wfchen-umich/relative_depth
7.Multi-Scale Continuous CRFs as Sequential Deep Networks for Monocular Depth Estimation
https://arxiv.org/pdf/1704.02157.pdf
https://github.com/danxuhk/ContinuousCRF-CNN
选用
http://perso.ensta-paristech.fr/%7Epinard/depthnet/ 1/4 downsampling,缺失训练代码,可以预测
https://hal.archives-ouvertes.fr/hal-01587652/document
https://github.com/ClementPinard/DepthNet
Depth Map Prediction from a Single Image using a Multi-Scale Deep Network 输出74x75 ,提供评估方法 https://www.cs.nyu.edu/~deigen/depth/
https://github.com/MasazI/cnn_depth_tensorflow 不完整
https://github.com/janivanecky/Depth-Estimation caffe
Qualitative Evaluation,质化方法,点云图像化展示
depth to (x,y,z)点云array 快速转换:https://codereview.stackexchange.com/questions/79032/generating-a-3d-point-cloud
点云可视化
https://github.com/daavoo/pyntcloud
https://github.com/createamind/busyplan/blob/master/zhangwei/point_cloud.ipynb
前期研究证明p2p模型可以利用虚拟vkitti数据集预测深度,现在决定继续深入, 目的如下:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.