Comments (11)
@hickeng, I had a discussion w/ the Lifecycle team regarding this, and received this response "you probably want to directly consume bits of disklib, ala diskTool. Writing useful state into the VMDK requires some tool that understands ext4. I vaguely recall the tools team having some code that inspects VMDK content invasively, but don't know the details"
Further asking about ext4 seems to indicate that there's no ext4 partitioning available on ESX, however this sort of thing could be done from inside of a Linux VM (I'm sure this is not surprising). Further discussion with Mike Hall seems to indicate that we might be out of luck as far as doing this in an application running directly on ESX but a service VM (or maybe just a process running on the VCH) should be able to perform these actions in a way that's more efficient/performant than our current approach, through the use of loopback block devices.
I tend to agree that performing these operations on the host directly would at the very least be considerably more challenging than performing them inside of a VM.
from vic.
We're currently performing the manipulation inside a VM, and that introduces either bottlenecks or significant throughput degradation; the latter could just require optimization, that's TBD.
Presenting the disk as a true block device in the VM requires attach/detach operations, which are serial and expensive as they entail a fast-suspend-resume, and you are limited to 4 scsi controllers and 15 disks per controller. You could partially address the serial element with clever operation batching, but the latency isn't going to improve.
If you mount the datastore directly (via 9p for example) then you bypass the need for attach/detach (see BON-26) however throughput was significantly lower. If the throughput performance could be improved and the security implications addressed then this latter approach becomes viable.
from vic.
Yeah, @mlh78750 and I discussed the current drawbacks with the current 'true block device' approach, @mlh78750 can you weigh in on this discussion?
from vic.
The thought I had was could we use something disklib and create a VMDK on a normal filesystem on a guest. Then loopback mount it as a block device and use the normal linux tools to format and lay the archives on that filesystem. Then unmount and move that VMDK (file) to the datastore.
from vic.
@mlh78750 loopback to what? You create the vmdk in the hypervisor or in the guest? If in the guest, sure, but is there a driver for that which doesn't rely on FUSE?
from vic.
I was under the assumption we had no support to format ext4 and mount it on the host. If that is not the case and we can, then that is a better solution. If we cannot, then yes, I was thinking of something using FUSE to mount that image.
from vic.
@hickeng do you have a pointer to the bonneville 9p code? I recall there are bits that live in the daemon (but could be wrong) and bits that live in esx (but could be wrong). A pointer to both/either would be appreciated.
from vic.
This looks mostly like BON-147, BON-148, and BON-149. FYI in case we want to close those jira issues as this should probably subsume them.
from vic.
Parent issue: #38
from vic.
Is this still relevant? @hickeng
from vic.
@gigawhitlocks still relevant, just not in plan for the near future - this requires an ESX agent and should be deferred until we have got the VMCI/vSocket communication work in place (that will supply the necessary basics and is higher priority).
from vic.
Related Issues (20)
- Networking issues with Containers HOT 7
- Registry whitelist
- VCH Configure to add new registry whitelist option missing HOT 1
- import settings.yaml somehow? HOT 2
- Cannot copy files to volume: error writing files to device HOT 1
- VIC Proxy for registry HOT 2
- docker stats API response not matching with actual usage in container HOT 7
- Splunk Logging Driver
- Precreated docker volumes are removed on an NFS volume store when the linked application is destroyed
- VIC Hostname Set Fails After Power Loss HOT 6
- VIC / VCH Shows No Existing Volumes HOT 1
- VCH Password expired. HOT 1
- Registry whitelist check failed for image mcr.microsoft.com/mssql/server:latest
- Error response from daemon: Server error from portlayer: Cannot reserve IP range Already in use
- Unable to add Global Registry
- docker: Error response from daemon: No such container: No such image:
- VIC Appliance API 502 Bad Gateway HOT 3
- Segmentation Fault error vic-machine-darwin
- End of General Support isn't well communicated. HOT 4
- VCH logs bundle only cannot be opened or extracted
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from vic.