JSON & plain typeNums might not work correctly (e.g. fluid styled content) about aimeos-typo3

[aimeos]

We have the same problem with the bootstrap_package which adds a div around the output in the 7.0.x versions.

How can we prevent that with typeNums? There are already to types defined for this case:
https://github.com/aimeos/aimeos-typo3/blob/master/Configuration/TypoScript/setup.txt#L90

[AndreasA]

2 questions:
1.) Do this typeNums require the content to be on a page or would it be enough that the always output a certain aimeos plugin?

2.) If it is better that they output the content of the page itself, would it be enough if it only outputs plugins (no text & media, text with images etc.)?

If one of those is fine I will create a PR (just let me know which). Hopefully this weekend.

[aimeos]

Only the output of the plugin is required. The page is only a helper and irrelevant.

[AndreasA]

Which plugins are those. It is definitely the stock plugin. The catalog count and the catalog suggest.
The suggest is used for
Any others?

Though probably easier to implement for now might be number 2. I will have to check.

In that regard for the basket add it would make sense to also create a typeNum that does not output the header etc. However, as it should also work without Javascript I would say that URL variant should be added as data-attribute.

[aimeos]

Only these three (catalog stock, suggest, count)

[AndreasA]

Sorry for the delay but had to finish our website relaunch first.
Which is now done - of course - there will be some further changes - so I should have time to do some of those things in the following weekends.

[aimeos]

Any news on this issue?

[AndreasA]

Sorry, for the delay was too occupied lately.
However, the weekend from 27-29 is completely free currently, so I should be able to tackle this issue then.
Maybe earlier but I cannot guarantee prior to that time.

[gilbertsoft]

Today I investigated this issue. The problem cames from the bootstrap / fluid packages. I have found a nicer solution for Ajax and Typo3: eID. Instead of working with page types it's possible to use eIDs. The great thing with eIDs is that not the whole core has to be loaded and of course that results to faster responses. I have found cool stuff and examples on this and my first tests were positive. Now I have to cleanup the code and do some changes at the .js files. How long will the Typo3 6.x support remain? There's the need of some special handling for this! Think should be finished tomorrow...

[gilbertsoft]

Here some links:
http://stackoverflow.com/questions/21139769/typo3-extbase-ajax-without-page-typenum
http://www.typo3-tutorials.org/cms/eid-mechanismus.html

[AndreasA]

Hmm. eID would probably be the better way to go. However, the links will have to be handled differently then. However, it shouldn't be necessary to add own pages for stock etc. as the eID should directly invoke the script.

The probelm is not really the fluid packages but that currently the plugins are rendered normally instead of directly (which could be done by using typeNum too but eID is better) or by ensuring that only the extbase plugin code is rendered when rendering those typeNums.

But be careful when executing plugins with eID, you need to ensure TypoScript and language is correctly loaded (which of course makes it a bit slower again then using eID without it but it is necessary here to ensure the correct settings)

P.S. Great that you found time for it, I really thought I would finally be able to tackle the issue but...

[aimeos]

We need to support 6.2 at least one more year, so 2018 we might be able to remove support for new versions.

If you need to change the JS, please keep in mind that it have to work not only with TYPO3 but also with all other integrations.

[gilbertsoft]

I'm going to push my solution. Here some informations:

• 6.2 should be supported, I used almost the same init as in scheduler, but only tested on 7.6!
• no JS changed, only t3 package files
• new partial constant editor support

All by the affected plugins used page ids had to be set in the constant editor, the plugin settings seems not to be loaded and I don't known if this is easy to do or not. Currently even empty settings of the plugins have higher priority than the constants from the constant editor. But that does not affect the new dispatcher and existing running installations. Checkout update is not tested, will do this tomorrow...

[gilbertsoft]

fyi, at the master branch of the boostap_package the gone frame type none (id 66) went back. But now there is a a-tag added, so is no solution for this problem. And changing the corresponding html-template is not a good solution I think.

I did not test, but I think should be possible without a page for this plugins now! Because the pid is loaded from the constants and not longer from the plugin.

[gilbertsoft]

Was a little bit late yesterday, I forget to push the changes in ai-typo3 extension. There is a little change to support th eID dispatcher. And a little bugfix here ;-)

[gilbertsoft]

I played around with removing the pages / plugins. It's possible but not so easy now because the existing links on other plugins. For example if you remove the suggest plugin you have to remove the link on the filter plugin too which is not possible without delete it directly in the database at the moment. I'm looking for a solution to remove existing fields in flexforms and their values from the db...

[gilbertsoft]

Ok, found something: will try it with a datahandler see http://www.derhansen.de/2014/06/typo3-how-to-prevent-empty-flexform.html

[AndreasA]

Isn't the link in the filter plugin defined by flexform? If so, there should be a delete button.
Or do you mean that it isn't correctly unset and therefore still overrides the aimeos configuration and the target is not null?
If so, this might be a general problem actually.

However, you should also consider if it might not make sense to remove those pages completely from the Flexform settings too.
If so, I would add the removal of those removed settings from Flexform to the Upgrade-Button in the Extension-Manager.
It would require to parse, remove wrong / not needed data and restore all flexform data for aimeos plugins.
Flexform can be parsed to array by using
TYPO3\CMS\Core\Utility\GeneralUtility::xml2array and
TYPO3\CMS\Core\Utility\GeneralUtility::array2xml to recreate the array.

There actually is also a Flexform cleanup command line tool (that removes not existing flexform fields from the databse) but I doubt that users will execute them by themselves. And it will also affect other plugins.

[gilbertsoft]

Yes, there is a delete button, but the user has to open each plugin which refers a deleted page and save it to get the invalid links deleted.

That's exactly what I did today ;-) You can see it at the updated PR. At the moment it does too much I think.

I'm currently defining the exact behaviour for updates and new installs including the aimeos_dist and possible later developments and optimizations. Later on more...

[gilbertsoft]

As mentioned above I pushed new files. Here what I did in short:

• removed the 4 plugins from ext_tables to get it out of the BE plugin list
• removed flexform defs of not longer used plugins
• rewrote code of settings loading at abstract controller to ensure ts setup settings are not overridden by empty values of plugin settings
• changed fields pointing to old plugin pages from mandatory to optional
• prepared some stuff to be able to cleanup flexforms at BE editing

Here my desired behaviour on new installs and updates and almost the current implementation:

Extension aimeos (without dist)

On new installs:

• install extension at extension manager as before
• configuration can be done at constant editor, plugin configuration isn't required anymore
• special configuration directly in ts setup or config files as before

So there's no change except the comfortable configuration at constant editor

On updates:

Minimal:

• update extension at extension manager as before
• run update script as before
• set constants "settings.client.html.catalog.detail.url.target" and "settings.client.html.checkout.confirm.url.target" at the constant editor at categorie "AIMEOS WEB SHOP: BASIC"

Recommended:

• set other constants at the constant editor in categorie "AIMEOS WEB SHOP: BASIC" to ensure everything works fine even no settings are made at the plugins

Optional:

• set other constants at the constant editor in other categories
• delete the pages Search, Count, Stock and Update. Open and save plugins catalog-list, catalog-detail, catalog-filter and checkout-standard to correct invalid references to the deleted pages

Extension aimeos_dist

On new installs:

• install extension at extension manager as before

On updates:

• update extension at extension manager as before

All significant changes are done by the update script as before including delete unsed pages and plugins. TS setup has to be cleaned up and adapted to the new constants.txt of aimeos ext.

To answer the questions and suggestions from above some other words:
I thought about deleting pages in the update script, but I think we could not guarantee that every installation has the same pages and layout. Think about multishop installations. Planed for the dist ext only for the moment.
I think my solutions for the problem with the flex forms empty values and the cleaning possibilities should be good enough, affects only aimeos and can be simply extended in the future.

Other toughts:
The target links from other plugins to this 4 deleted ones can stay in the flexforms for backward compatibility. If the user deletes such pages he has to adapt the settings by open the plugin and save it again. BE will autocorrect the missing link to the deleted page. Theoretically could by automated by a db update hook on deleting pages. But I think this is only timewasting for the moment and slows down page deletions at the BE.

Todo's:

• change to the described behaviour above, will do this next
• update docu for updates with important actions mentioned above
• add docu / hints to the new used constant editor feature

Any suggestions / remarks?

[gilbertsoft]

interessting articles: http://insight.helhum.io/post/104880845705/dont-use-eid-ajax-dispatchers-for-your-extbase

[aimeos]

There are many multi-domain/multi-pagetree installations using the Aimeos plugins several times with different targets. Therefore, a constant editor seems to be not that useful.

Deleting the old pages for stock, count, etc. is likely to fail, so it's no option to do that at all. Removing the references to this pages in the Aimeos flexforms stored in existing databases would be mandatory instead, so there are no wrong references any more.

[gilbertsoft]

Ok, but are they in the same root template or different roots? I'm talking about the page "Your Shop" of the dist with the root template record.

[aimeos]

That depends on the installation. Some use different root templates, some share the same and there it has no advantages and maybe some disadvantages.

I think there's a subset of settings where the constant editor is very useful, e.g. if there's a default setting like for the Aimeos CSS/JS. For the PIDs I don't think so because it's simpler for editors to select the target page from the plugin backend. Also the settings for developers only (jQuery) are not so useful. The plugin namespace it's a parameter that can't be changed at all because at other locations it's hardcoded and it doesn't make much sense to change it at all.

Nevertheless, we would like to get a simplified version of your PR into the Aimeos TYPO3 extension :-)

[AndreasA]

@gilbertsoft
Maybe you could also try using typoscript_rendering because I think it would actually make sense to use it. From what I understand one of the main advantages is that it doesn't require manual TSFE initialization.

However, the eID approach as long as it works correctly is fine too. The typoscript_rendering approach just isn't slower and only has advantages if one uses caching for the Ajax call.
Well, of course there is also the "update" stability.
No need to check if TSFE initialization still works with newer TYPO3 versions.

[gilbertsoft]

@AndreasA
Yes, I already did this, but tests are not finished now....

And yes, I think too that it makes sense to use it. After reading the articles of Helmut Hummel I think he really knows what he's talking about!

[gilbertsoft]

@aimeos
Perhaps it's better to continue this discussion about the constant editor feature at the relevant PR?

As you see there are several categories at the constant editor. I added the hints of the t3 docu to the top of the constants.txt for the moment. Important settings are found in cat basic, so if you don't think page id are important enough I could change it to advanced. There is also the possibility to write a user function to get a nice page selector at constant editor, but I didn't have the time for that.

With this new knowledge I will try to adapt the constants.txt...

[aimeos]

@gilbertsoft What happend to the eID changes? I think this is the more important PR to get around the bootstrap_package problem. Can you open a separate PR for them based on the current commit in the master branch of this repository?

[gilbertsoft]

As you can see above there's a cool extension from Helmut Hummel (typoscript_rendering) which allows to do such things I did with my eID dispatcher but with the benefits of t3 caching. So I stopped working on the eID dispatcher and switched to this ext. Implementation is done, but had no time to test it. But don't worry, for me this is the most imprtant thing in my customers webshop because I like to start with the newest bootstrap ext. I will open a new PR from my new branch, but as said, not tested!

[gilbertsoft]

Got news on typoscript_rendering: requires a cHash, without you only get an exception back! 2 of the 4 cases can probably be implemented, catalog-suggest and checkout-update surly not because there are parameter changes at client side.
So work goes back to eID dispatcher...

[AndreasA]

You can disable cHash for certain parts usually. Or exclude parameters from cHash.

Might require your own renderer that disables caching because otherwise yes, it requires cHash.
But actually the URL-Generator should be able to add cHash if it has typoscript_rendering parameters.

[gilbertsoft]

The problem is not in the uriBuilder, the URLs are generated correctly as I see. But if you omitt the cHash parameter the extension throws an exception. Maybe it's possible to exclude these client side parameters from hashing as you write, but I think then we get wrong result from the cache. I never cared about the caching framework and how it works in deep and which the possibilities and where the limit are. So I only could make a PR on the current code base to a new branch and perhaps someone finds the time to play around a little bit more...
The whole Ajax at the backend is implemented by eID and for the newest version I copied some nice code snippets from there, will become a nice solution I think!

[AndreasA]

Actually yes it is the UriBuilder because the UriBuilder should be able to add a correct cHash for URLs that require a cHash. Which it doesn't currently.
I don't mean the TYPO3 UriBuilder but the Mapper built into aimeos extension.

In regards to caching:
You would have to make your ajax request uncachable which I think they should be anyway in this case. It doesn't make sense to make suggest, stock, ... cachable.

In regards to those parameters, they need to be added like this (should use both):
// TYPO3 6.x
$GLOBALS['TYPO3_CONF_VARS']['FE']['cHashExcludedParameters'] .= ',ai[parameter1]';
// TYPO3 7.x
$GLOBALS['TYPO3_CONF_VARS']['FE']['cacheHash']['excludedParameters'][] = 'ai[parameter2]';

needs to be called in ext_localconf.php

If the called Extbase plugin is defined as uncachable this should work fine. Not sure which parameters would be needed to be included.

On the other hand using eID should work fine too, so it doesn't really matter that much.

EDIT:
Your own Parameters would have to be added that way.
However, the typoscript_rendering parameters should not be added.
This would require a change to the UriBuilder to include the cHash for URLs generated to target typoscript_rendering.
That way the important typoscript_rendering Information would include the cHash but your parameters that are appended by using Ajax will be excluded.
But as mentioned when using non aimeos Parameters the Uri-Builder would need to include the cHash which it is capable of (only would have to execute setUseCacheHash(true) or similar).
I think one could check for this if the URL includes typoscript_rendering parameters then include cacheHash, otherwise don't.

Another way would be to add all aimeos Parameters to the excluded parameters and always include the cacheHash (will not be added if no parameters are found that require it)

The point here is that most of the request would be cached this way and only the aimeos plugin would be inserted uncached then.
TYPO3 does take care of things like this, e.g. if you had a page with a plugin that is cachable and one that is not cachable, the cachable plugin would be in the cache and only the uncachable one would be re-rendered on the second request and inserted.

EDIT2: Actually, I think the master branch is already supporting cHash with the Uri-Builder now, so only the exclude should be necessary and setting the corresponding plugin to uncachable which they already are.

[gilbertsoft]

@AndreasA : Thanks for the infos!

@ALL: finally the eID solution is almost finished. But there is still a big bug, in this eID context TYPO3 always asumes the root page as source in uriBuilder. So I always get wrong URIs for the catalog-suggest. Work around was to set absoluteUri option of uriBuilder. Overridding settings with the plugin was not longer possible, because I never get this setitngs. But then I started to debug a problem on 6.2 and therefore I have compared the TSFE initialization of 6.2 with 7.6. I was shocked to see that my eID implementation calls almost every initialization as the TypoScriptFrontendController does. The response time of eID and pagetype request was almost the same. So I decided to change the ts setup to enforce the rendering without fluid. Now everything works fine!

@aimeos: some thoughts on an possible eID solution: to profit of the smaller footprint of eID, the access to the data has to be more direct and without the need of the frontend. It may be possible to direct involve the core without the html-client, so there should not longer be a dependency to the frontend. But this would primary be a TYPO3 solution an independent from other frameworks.

[aimeos]

The problem when using the bootstrap_package 7.x should be fixed by this commit: https://github.com/aimeos/aimeos-typo3/pull/30/files

As the eID stuff has no major advantage (according to @gilbertsoft), we can still use pages for JSON responses.