Comments (48)
Well, that's the problem :) we don't have those files on the appliance at all. All we have is /var/www/miq/vmdb/public/self_service/js/app-*.js
and lib-*.js
, which are respectively the concatenation of all the files from client/**/*.js
and the concatenation of all the library dependencies. That's it :).
That's why I was proposing having a copy of the ssui source on the appliance, and having all that's needed to regenerate these app*
and lib*
files from that source.
from manageiq-ui-service.
@himdel so, some thoughts to your comments:
RE: frequency of hotfixing
I can't comment about how rare hotfixing is, but it seems that it is needed enough for @dclarizio to request a system to manage it (whatever that system is).
RE: security
I agree with you there, we do open a possible vector, but it seems like a good point to start a discussion for something to do.
from manageiq-ui-service.
I'll go with option 2 also since it seems like the most feasible option at the moment.
Maybe we could deploy a tool like appliance_console
that we currently have on our appliances to apply the hotfix and rebuild SSUI.
from manageiq-ui-service.
Adding @martinpovolny @AparnaKarve @himdel for thoughts
from manageiq-ui-service.
With all the new things we are doing, I wonder if the rails asset pipeline is going to continue to cause issues; it really doesn't help us (the SUI team) much.
from manageiq-ui-service.
@eclarizio and I went through this once... for sui because we cat everything into 4 or so files, ja just gotta rebuild the file in question (but really only two, because two of the files are dependency files)
from manageiq-ui-service.
@AllenBW wrong "clarizio"
from manageiq-ui-service.
Nope, right one :-P, roping him in to see if this ended up working in the end or what became of our fix...
from manageiq-ui-service.
Well, moving to a separate node app means it would be easy. But that's not something we can backport, however, we can do something similar for euwe & darga..
- ensure the appliance contains a copy of the self-service repo
- ensure the appliance contains all the dependencies for
gulp build
- then, to hotfix we simply patch the repo copy (or untar the changes over it, same as in the ops ui)
- aand run
gulp build
(and possiblynpm install
for hotfixes containing dependency changes, if any)
That way, it works the same whether we just have that copy or a running node app, it works the same as in the ops ui, hotfixes can be stacked (with care - obviously that means patching, not replacing), and everything works with minimal changes to the existing process :).
EDIT: though note that to support hotfixes with dependency changes, we'd need to install a newer npm (ie. run npm install -g npm
during the appliance build process or before the first patch).
from manageiq-ui-service.
@himdel You are right, this really isn't something we can backport. I'm not sure I fully understand your solution. We are trying to get away from from doing what you are describing (again, not sure I fully understand). That way we can provide the customer with the ability to drop in a file, and restart the server. There would be nothing more for them to do.
from manageiq-ui-service.
@chriskacerguis dropping a file and restarting the server works only if the file is a file containing all the javascript from SSUI, concatenated (and you don't actually have to restart the server in that case).
That means we would never be able to do 2 hotfixes on one machine (because you have no way to even find out if a hotfix had been applied or not), and certainly makes the process different from what is done in the ops ui - replacing only the changed source files.
My solution means you can do exactly that - replace the source files that need changing, it also means you can find out if a hotfix had been applied or not (diffing against the release source), and you can apply as many hotfixes as you want (that one only provided you patch the files instead of replacing, or that there's no overlap).
It also means that if we ever do move towards node serving stuff on the appliance, it will continue to work exactly the same, because you'd still be replacing the source files. (There will be a difference in that before node server, you run gulp build
after applying the patch, after node server, you'd restart the node server after applying the patch instead.)
So.. depends on the priorities.. If the goal is indeed to just give the customer a file and nothing else, then what we already have works that way :). But the whole reason we're having this discussion is that we can't just expect to replace the built artifact and be done with it because it breaks the moment you need more than one hotfix, it breaks with any custom local changes, etc.., isn't it?
from manageiq-ui-service.
@dclarizio To give some context here, can you please chime in with the details of what is prompting this?
from manageiq-ui-service.
@himdel I'm not sure I understand you when you say "dropping a file and restarting the server works only if the file is a file containing all the javascript from SSUI". Wouldn't that file just need to contain the contents of the replaced file?
from manageiq-ui-service.
@himdel thank you for clarifying. That is what I thought was being done here #217 by the rails asset pipeline. So, as a part of that, we need to stop doing that concatenation.
from manageiq-ui-service.
@chriskacerguis No, we cannot stop concatenating files, we need to serve these files concatenated in production - otherwise, you'd be downloading ~200 files on each load, instead of 2. That's why, to be able to hotfix, we need to be able to do the concatenating on the appliance itself.
from manageiq-ui-service.
@himdel I wonder if we need to rethink that. I would want to dig in more regarding why we would need to load ~200 files. One of the big questions that I have is everything we are loading required, or are we loading things that wouldn't necessarily be required? I agree that the less http requests the faster the app, but if we start to break things up wouldn't that allow the browser to cache a bit more efficiently?
I tend to break things up in 3 way:
- Things that are needed on every page
- Things that are needed per module
- Things that are needed per page
If we move this to a task, there is a lot of work for sure. Another thing we need to consider is do the pros outweigh the cons (multiple files make for easy hotfixing, but bring up all the hints that you have mentioned).
Finally, for the time being we can't run gulp build on the appliance as it doesn't have Node 6.x. I'm sure it will soon as it is now LTS.
from manageiq-ui-service.
@chriskacerguis Well, 122 files is our source, and we're using about 47 external dependencies, so at least that many files, probably more :).
Loading things that are not necessarily required is a moot point, sorry, angular 1 does not support lazy loading, so we're out of luck. Angular 2 does however, so we could revisit that then. (And I have heard about projects doing their own lazy-loading in angular 1, but it was always described as an ugly hack which doesn't really give me any confidence.)
As for gulp build
, you're right, but for darga, we definitely don't need node 6, and as for euwe, I guess @AllenBW knows better than me, but imho the only thing requiring node 6 was eslint which we don't have to use during gulp build, and node-sass which doesn't have to go in euwe, right?
from manageiq-ui-service.
Actually most things can be made happy with node 4 (eslint is >=4) but if we're going to update, why not update (unless it breaks something unfixable). I say this not knowing the context/direction of the conversation, don't mind me.
from manageiq-ui-service.
@himdel yeah, again there's lots of work to be done...and not a "simple" thing. I know Angular 2 does lazy loading, and I think that is something we will want to look at eventually.
from manageiq-ui-service.
So the suggestion that @himdel made earlier about possibly using gulp-build on the appliance would be the best route to make this possible. The practice we are using of taking all our source files and combining them for ease of use at runtime is a fairly common industry practice these days. We could actually go a similar direction and generate new JS files with the hotfix for the user , maybe as a zip file and then have the user upload the zip file via a web interface. If we started using cache busting gulp plugins, we could make sure the new concatenated JS files have a different filename automatically than the old ones and just repoint index.html to the new js files.
Just a thought
from manageiq-ui-service.
we could make sure the new concatenated JS files have a different filename
@chalettu the generated files already have a hash in the name, so.. yeah, we already do that :).
from manageiq-ui-service.
Cool. Didn't look close enough to see that, well that makes that easier then.
from manageiq-ui-service.
@himdel so, I think @chalettu might be on to something. I think that the goal is to make hot fixing as easy as possible for the end-user. So, any suggestions on making that easy are welcome to get this requirement moving...and I like @chalettu thought on that (some sort of upload tool)
from manageiq-ui-service.
Aaah, sorry @chalettu, I completely missed the upload suggestion.
Well, yeah, from the user's perspective, this is a great idea! I'm not as sure if it's still great given the context though - 2 things concern me.. a) hotfixing is (AFAIK) relatively rare, so it feels like making a web UI for that could be an overkill, especially since who'd be applying the patches would be sysadmins anyway, right?
And b), security. By implementing something like that, we're opening a huge possible attack vector, since the app needs to be able to write in it's own directory without restriction, and also to the manageiq (or at least manageiq/public
) directory, and to run scripts in the system. Which means the impact of any security oversight in SSUI would essentially change from a mild inconvenience to a possibility of complete control of the appliance.
So.. it's a trade off, and I'd say, if we expect people to do that daily, sure, go for it, but it seems a disproportionate amount of work for something used rarely. (Also, this is something we can't do for darga or euwe retrospectively.)
from manageiq-ui-service.
Just to make sure we are all on the same page, here are the requirements that I have been given:
- This "hotfix" system would be for the "F release" or above
- Need way for customer to easily apply hotfixes (and be able to apply hotfix on top of hotfix)
- Build tools (e.g. Node 6.x) will not be available to customer.
@dclarizio did I miss any?
Current (high level) solutions from this thread (if I missed one, please let me know).
Proposed Solution 1
Break up the JS files into the individual files (do not concatenate), create a system in which the system will look for a "hotfix" file first, and if it doesn't fine one, then it will load the stock file.
- CON: that's a lot of files to load (would need to move to a lazy load like system)
- PRO: easiest way to hotfix the system
Proposed Solution 2
Keep a copy of the source files on the appliance, if a hotfix is needed, put the new file in the source, and "rebuild" the SSUI as needed
- CON: Build tools may not be available on the appliance
- PRO: Simplest solution from engineering, could be complicated for end-user
Proposed Solution 3
Add some form of web UI to manage hotfixes that would upload a new file, then reload the server.
- CON: Possibly opens a security hole
- PRO: Would allow for a robust hotfix / web-update solution
From where I sit, I like option 3 the best (the web-based updater, not the hotfix system). My 2nd choice would be option 1.
@dclarizio @AllenBW @chalettu @himdel @martinpovolny @AparnaKarve what's your thoughts on which solution to go with (or feel free to propose a new solution that meets the requirements)?
/cc @chessbyte @Fryguy @bascar
from manageiq-ui-service.
Aaah, IMO requirement n.3 (build tools not available) severely limits what we can do ... essentially down to only solution 1, so I think that one needs to be relaxed.
I'm for solution 2 (4.), mainly because I think solution 1 is unrealistic in production, and 3 (7.) feels like a possible overkill, and still needs to use either 1 or 2 under the hood.
from manageiq-ui-service.
@himdel I very much agree with you, but I can't do anything about the build tools. Node 6.x is not available, and so until Node is updated to 6.x in the "blessed" repos we can't add it to the appliance.
from manageiq-ui-service.
tbh I'm still hoping it will get RPM'd before the F release.. but if not, then yes, we do have a problem. (I'd say just install it manually when doing the first hotfix, but not at all sure how feasible that really is..)
from manageiq-ui-service.
I vote for the 3rd option. There are a lot of ways you could minimize an
attack vector to make it safer. My second choice would be option 2. I
would have to learn a little more about how we package our codebase for use
on appliances before talking more about it.
On Wed, Oct 5, 2016 at 6:13 AM, Martin Hradil [email protected]
wrote:
tbh I'm still hoping it will get RPM'd before the F release.. but if
not, then yes, we do have a problem. (I'd say just install it manually when
doing the first hotfix, but not at all sure how feasible that really is..)—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#219 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAOElRhqd48HQvNdP3jWJGEqcJ7EVuk-ks5qw4ZogaJpZM4KNAeq
.
Chris Hale
Senior Software Engineer
[email protected]
512-585-7286
https://bluejeans.com/chhale
from manageiq-ui-service.
I like option 2 the best for many reasons but live troubleshooting/screensharing/fixing corner case issues on the fly and testing immediately is the top of the list.
Hotfix on hotfix is also a concern. Any way we could use git/hashing/other as a way to verify that what we are laying down as a fix is in fact loading into the "known" environment we expect the fix to lay into?
from manageiq-ui-service.
@bascar with option 2 we are limited until we can get Node 6.x (which hopefully will be QUICK) on the appliance, but that's not up to our team. That said, what option do you like?
from manageiq-ui-service.
Option 3 has an issue, load balancers/multiple appliances. The customer would have to upload that file to every UI appliance that has our source code, from the Web UI. Granted, this issue does exist even for server based hot fixes, so perhaps we can build that into the Web UI screens somehow.
I'm for option 2, which follows more closely how we do existing hot fixes, but simply adds a script step at the end, which could hopefully be automated.
from manageiq-ui-service.
@dclarizio I think with option 2 we could have a system setup that as we update the "branch" in Git, it would auto-update from that branch. So, if they were running E release then every time we updated the E branch it would pull down the latest (maybe even have an auto-update script that runs), so it would update all the UI's.
from manageiq-ui-service.
@chriskacerguis I think I still like option 2 :)
What is the dependency on Node 6? Can't we carry our own packages for it until they are pulled into the "blessed repos"?
from manageiq-ui-service.
@bascar As I understand things (which could be incorrect) for the appliance we can only pull from a "blessed" repo, at the present time only Node 4.x is available. We cannot add the NodeJS rpm repo. @simaishi could you please confirm / clarify things for me and @bascar?
[edit] forgot the word "rpm"
from manageiq-ui-service.
@chriskacerguis @bascar For downstream, we can only install RPMs from the "blessed" repos. We can package our own and use that, but that means we have to maintain the RPM as well. We had some discussion about this in email and I think we decided not to build our own. cc @Fryguy @chessbyte
from manageiq-ui-service.
Thanks @simaishi !!!!!!
from manageiq-ui-service.
@Fryguy any concerns from your end about doing "Proposed Solution 3", and basically make a web updater?
from manageiq-ui-service.
To start I'd clarify the requirements:
- This "hotfix" system would be for the "F release" or above
- Need way for customer to easily apply hotfixes (and be able to apply hotfix on top of hotfix)
- Build tools (e.g. Node 6.x) will not be available to customer.
From the above only 1. is actually true.
Number 2.: hotfixing this way should be very rare (we have RPMs or DEB packages do to easy upgrades and the person doing the hotfixing has to be an experienced sysadmin not an accountant.
Number 3.: as was written earlier in the discussion we don't need the latest Node for the build.
On solutions:
Solution number 1 was rejected due to technical reasons. We don't want the files to be split nor does our toolchain and used libraries support that.
Solution number 3 is an overkill. We are not writing software for home modem where end user needs to upload fixes. We actually distribute appliances and in Linux distributions we distribute packages. Also we do not intend to support combinations of hotfixes. We actually do not want so support hot-fixed installs except for some rare exceptions.
That leaves us with Solution 2 that I personally find the only reasonable and feasible one.
from manageiq-ui-service.
@chriskacerguis Option 3 does seem like overkill to me as well. We would be investing in something that we would not need once Node 6 is on the appliance. Additionally, the Web UI of Option 3 would need to make 2 big assumptions: 1) only allow system admin to do this and 2) all CloudForms appliances may live inside the corporate firewall and have no "internet" access. So, I vote for Option 2, aka KISS
from manageiq-ui-service.
@martinpovolny because we have updated the dependancies, we need Node 6.x. As we saw, when Node was running an old version, the build failed. Also, those requirements are neither true nor false, they were communicated to me; they simply just are (their validity may be questionable).
@chessbyte I agree an update system for just hot fixes is overkill, however from an overall process I do think that an auto-update system could be beneficial for the project; but I think that is a larger discussion.
So, as it stands, Node 6.x is not available on the appliance; so until then we will just have to build files for a hot fix until Node 6.x is available. I will leave it in the very capable hands of @simaishi to get that figured out. Once it is available, we will go with Solution 2.
from manageiq-ui-service.
Re-opening this for additional discussion.
from manageiq-ui-service.
@chriskacerguis We will also need to find a way to deploy this differently for downstream (probably RPM, but we are still discussing), so whatever choice is made should be aware of that.
from manageiq-ui-service.
Reading back, option 2 is probably the most flexible and allows us to deploy hotfixes via RPM downstream (and with RPM allowing us to auto-build the SSUI afterwards).
One downside to the rebuilding is how we deploy the files downstream anyway though. We pre-build the SSUI files and make them part of the RPM. If we remove and replace them, then I think RPM will not be happy. We'll have to find a way to avoid that, perhaps by not pre-building in the first place. cc @simaishi
from manageiq-ui-service.
@Fryguy Was that Martin P's "option 2", b/c option 2 from my list was keeping a copy of the source on the machine and rebuilding as needed.
from manageiq-ui-service.
@chriskacerguis I was referring to proposed solution 2 from here #219 (comment)
from manageiq-ui-service.
cools. just wanted to make sure :)
from manageiq-ui-service.
@Fryguy If we go with option 2, then that's more of a system management, not really a "code" issue per se. So, going to close this issue out and we can stick with option 2 (which I'm not sure is ideal, but keeping with KISS principles and "I don't have a better idea"™ it seems like the way to go)
from manageiq-ui-service.
Related Issues (20)
- CVE-2022-31129 (High) detected in moment-2.24.0.tgz - autoclosed HOT 1
- Upgrade or remove Angular code. HOT 3
- CVE-2022-25869 (Medium) detected in angular-1.8.3.tgz HOT 1
- WS-2022-0284 (High) detected in moment-timezone-0.4.1.tgz, moment-timezone-0.5.33.tgz HOT 3
- WS-2022-0280 (High) detected in moment-timezone-0.4.1.tgz HOT 1
- WS-2022-0322 (High) detected in d3-color-1.4.1.tgz HOT 4
- CVE-2022-3517 (High) detected in minimatch-3.0.4.tgz - autoclosed HOT 1
- prepare the development environment to make code changes HOT 5
- CVE-2022-24999 (High) detected in qs-6.5.2.tgz - autoclosed HOT 1
- Unable to open a Native Console for a RHV VM in the Service UI
- CVE-2022-25881 (Medium) detected in http-cache-semantics-4.1.0.tgz - autoclosed HOT 1
- CVE-2023-28155 (Medium) detected in request-2.88.2.tgz - autoclosed HOT 1
- CVE-2023-26117 (Medium) detected in angular-1.8.3.tgz HOT 1
- CVE-2023-26118 (Medium) detected in angular-1.8.3.tgz HOT 1
- CVE-2023-26116 (Medium) detected in angular-1.8.3.tgz HOT 1
- CVE-2022-25883 (High) detected in semver-7.3.8.tgz - autoclosed HOT 1
- CVE-2020-23064 (Medium) detected in jquery-3.4.1.tgz, jquery-3.2.1.tgz - autoclosed HOT 1
- Opening the service ui results in downloading app.js over and over HOT 2
- CVE-2024-21490 (High) detected in angular-1.8.3.tgz
- CVE-2023-42282 (Critical) detected in ip-2.0.0.tgz - autoclosed HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from manageiq-ui-service.