Very Long Directory Path about geometry

[awulkiew]

The fix is ready. As I mentioned above I've chosen the compromise solution. One link is not backward compatible. Currently the max filename length is set to 59 (can be changed with program option in python script). If there are additional problems in the future we'll apply more severe solution. The filenames for ctors are:

rtree__.html
rtree_ite_ite_all_cr_pac_cr.html
rtree_ite_ite_par_cr_ind_cr_val_cr_all_cr_pac_cr.html
rtree_iterator__iterator_.html
rtree_iterator__iterator__allocator_type_const___.html
rtree_par_cr_ind_cr_val_cr_all_cr.html
rtree_ran_cr_all_cr_pac_cr.html
rtree_ran_cr_par_cr_ind_cr_val_cr_all_cr_pac_cr.html
rtree_range_const___.html
rtree_range_const____allocator_type_const___.html
rtree_rtree____.html
rtree_rtree_____allocator_type_const___.html
rtree_rtree_const___.html
rtree_rtree_const____allocator_type_const___.html

[teeks99]

I did the initial look at the 1.60 version, and only noticed that single very long name. However in the 1.63 version there are a large number of files inside geometry with long paths, mostly within the doc\doxy\doxygen_output\html_by_doxygen directories, could these be shortened to doc\doxy\html? (and the xml output to doc\doxy\xml?)

Also some of this seems to bleed over into doc\index\html_by_doxygen and doc\index\xml.

[mloskot]

Those files should not be included in release packages. It was due to incomplete cleanup in docs generation process. It has been improved for 1.64. See #387

[garyfurnish]

Something to consider is that I think this is also in practice an issue with running tests on windows. This error doesn't seem to happen if I run it from the root directory...

d:\boost\libs\geometry\test>b2.exe address-model=64 -j8 > out.txt
failed to write output file '..\..\..\bin.v2\libs\geometry\test\algorithms\relational_operations\disjoint\algorithms_disjoint_point_box_geometry.test\msvc-14.1\debug\address-model-64\asynch-exceptions-on\threading-multi\algorithms_disjoint_point_box_geometry.exe.rsp'!

[mloskot]

@garyfurnish Yes, it is a common problem. The b2 --abbreviate-paths should help to workaround this.

@teeks99 I'm closing this. Feel free to reopen if there is still a problem.

[mloskot]

Hmm, looks like the long paths issue is hitting the docs again, @teeks99 reported https://lists.boost.org/boost-users/2020/07/90536.php

/cc @awulkiew

[teeks99]

so the original file that had the issue re-appeared. did the previous change for 1.64 get broken somehow?

[awulkiew]

Are you sure that the same file is causing problems? My guess is that it is caused by a similar file, also generated for rtree constructor, but a different one.

Recently several new rtree constructors were added (10b0c37), some of them taking many arguments. The tool automatically generating the documentation puts all of the parameter types in the name of a file. So my guess is that the path is probably something like: libs/geometry/doc/html/geometry/reference/spatial_indexes/boost__geometry__index__rtree/rtree_iterator__iterator__parameters_type_const____indexable_getter_const____value_equal_const____allocator_type_const____PackAlloc_const___.html

We should probably modify our utility doxygen_xml2qbk to generate shorter paths. This will not be backward compatible. So the link on the page "show the latest version of this file" will not work.

[teeks99]

Here's what boost_1_74_0_b1_rc1 looks like:

$ ls boost_1_74_0/libs/geometry/doc/html/geometry/reference/spatial_indexes/boost__geometry__index__rtree/            
begin__.html
bounds__.html
clear__.html
count_valueorindexable_const___.html
empty__.html
end__.html
get_allocator__.html
indexable_get__.html
insert_convertibleorrange_const___.html
insert_iterator__iterator_.html
insert_value_type_const___.html
operator__rtree_const___.html
operator__rtree____.html
parameters__.html
qbegin_predicates_const___.html
qend__.html
query_predicates_const____outiter_.html
remove_convertibleorrange_const___.html
remove_iterator__iterator_.html
remove_value_type_const___.html
_rtree__.html
rtree__.html
rtree_iterator__iterator__allocator_type_const___.html
rtree_iterator__iterator__allocator_type_const____packalloc_const___.html
rtree_iterator__iterator_.html
rtree_iterator__iterator__parameters_type_const____indexable_getter_const____value_equal_const____allocator_type_const____packalloc_const___.html
rtree_parameters_type_const____indexable_getter_const____value_equal_const____allocator_type_const___.html
rtree_range_const____allocator_type_const___.html
rtree_range_const____allocator_type_const____packalloc_const___.html
rtree_range_const___.html
rtree_range_const____parameters_type_const____indexable_getter_const____value_equal_const____allocator_type_const____packalloc_const___.html
rtree_rtree_____allocator_type_const___.html
rtree_rtree_const____allocator_type_const___.html
rtree_rtree_const___.html
rtree_rtree____.html
size__.html
swap_rtree___.html
value_eq__.html

vs. boost_1_73_0

$ ls boost_1_73_0/libs/geometry/doc/html/geometry/reference/spatial_indexes/boost__geometry__index__rtree/                                                                                                     
begin__.html
bounds__.html
clear__.html
count_valueorindexable_const___.html
empty__.html
end__.html
get_allocator__.html
indexable_get__.html
insert_convertibleorrange_const___.html
insert_iterator__iterator_.html
insert_value_type_const___.html
operator__rtree_const___.html
operator__rtree____.html
parameters__.html
qbegin_predicates_const___.html
qend__.html
query_predicates_const____outiter_.html
remove_convertibleorrange_const___.html
remove_iterator__iterator_.html
remove_value_type_const___.html
_rtree__.html
rtree__.html
rtree_iterator__iterator_.html
rtree_parameters_type_const____indexable_getter_const____value_equal_const____allocator_type_const___.html
rtree_range_const___.html
rtree_rtree_____allocator_type_const___.html
rtree_rtree_const____allocator_type_const___.html
rtree_rtree_const___.html
rtree_rtree____.html
size__.html
swap_rtree___.html
value_eq__.html

[mclow]

Just an observation: The new files (with very long names) have no backwards compatibility requirements (because they are new). So if someone were to rename them (somehow), and fix up the links in the generated docs, then we'd be golden.
(Obviously this would have to be done via some jam settings, since we don't want a one-off thing, but can we get doxygen to generate short file names for only some files?)

[awulkiew]

FYI, these names are generated automatically by quickbook when section name is full name of a function/ctor with parameters, etc. and there is no section id.

I have a patch for our tool ready which sets section ids explicitly and the result is this list:

126rtree.html
begin.html
bounds.html
clear.html
count_val_cr.html
empty.html
end.html
get_allocator.html
indexable_get.html
insert_con_cr.html
insert_ite_ite.html
insert_val_cr.html
operator61_rtr_cr.html
operator61_rtr_rr.html
parameters.html
qbegin_pre_cr.html
qend.html
query_pre_cr_out.html
remove_con_cr.html
remove_ite_ite.html
remove_val_cr.html
rtree.html
rtree_ite_ite.html
rtree_ite_ite_all_cr.html
rtree_ite_ite_all_cr_pac_cr.html
rtree_ite_ite_par_cr_ind_cr_val_cr_all_cr_pac_cr.html
rtree_par_cr_ind_cr_val_cr_all_cr.html
rtree_ran_cr.html
rtree_ran_cr_all_cr.html
rtree_ran_cr_all_cr_pac_cr.html
rtree_ran_cr_par_cr_ind_cr_val_cr_all_cr_pac_cr.html
rtree_rtr_cr.html
rtree_rtr_cr_all_cr.html
rtree_rtr_rr.html
rtree_rtr_rr_all_cr.html
size.html
swap_rtr_r.html
value_eq.html

Now, I could push it as it is and break all of the links in Spatial Index part of the reference or do this only for names longer than X. So the question is which option would be preferable? In case of the latter what name length would satisfy you?

[mclow]

Which are the new names in 1.74.0?

[mclow]

Looks like:

rtree_iterator__iterator__allocator_type_const___.html
rtree_iterator__iterator__allocator_type_const____packalloc_const___.html
rtree_iterator__iterator__parameters_type_const____indexable_getter_const____value_equal_const____allocator_type_const____packalloc_const___.html
rtree_range_const____allocator_type_const___.html
rtree_range_const____allocator_type_const____packalloc_const___.html
rtree_range_const____parameters_type_const____indexable_getter_const____value_equal_const____allocator_type_const____packalloc_const___.html

Those can be made shorter w/o breaking any links.
This one, though:
rtree_parameters_type_const____indexable_getter_const____value_equal_const____allocator_type_const___ is a problem

[awulkiew]

I'm fine with 1 non-working link. Are there any other ideas?

[mclow]

it's your library :-) (That's not a criticism of your opinion, just a statement that you are (one of) the owners of Boost.Geometry)

[awulkiew]

@barendgehrels @mloskot @vissarion opinions please. :)

[awulkiew]

@teeks99 What file length would be ok for you? Are the names generated for constructors the only ones that are too long?

[vissarion]

Now, I could push it as it is and break all of the links in Spatial Index part of the reference or do this only for names longer than X. So the question is which option would be preferable? In case of the latter what name length would satisfy you?

I am OK with both options. The one that does the job only for the longer filenames seems a bit preferable since it is less aggressive.

[teeks99]

To be clear, it isn't a filename length limitation, it is a full path limitation. On windows the limit has historically been 256 chars, this is being relaxed but not all tools support longer paths yet.

The above directory (102 chars):
boost_1_73_0/libs/geometry/doc/html/geometry/reference/spatial_indexes/boost__geometry__index__rtree/

Plus the file (146 chars):
rtree_iterator__iterator__parameters_type_const____indexable_getter_const____value_equal_const____allocator_type_const____packalloc_const___.html

Is still under this, but it was close enough that my D:\RB\build-msvc-10.0 (19 chars + D:\) was over the limit. Users may reasonably decide to install boost into C:\Program Files\Boost\ (21 chars), so I think we should limit paths to 256 - 21 = 235 chars. Shorter would be better.

Clearly shortening the very long filenames would help a lot, but there's also some redundancy in the directory path. Can we take the geometry inside html out? How about the boost__geometery__index part of the end directory? I realize that these would break the websites "go to the current version" feature...how critical is that?

[awulkiew]

How about the boost__geometery__index

This is also an option. Actually I've already implemented it locally to generate b_g_i_... instead. Removing the namespaces entirely is also an option which would probably work for now but in general could cause naming conflicts. But anyway, changing this would break backward compatibility for the rtree and its members, observers and parameters.

There is also an option of shortening spatial_indexes to index. This would break backward compatibility for all spatial-index-related pages.

Can we take the geometry inside html out?

This is generated by xsl/quickbook and can probably be set in Jamfile, maybe? I don't know. It may be related to some other setting like sections nesting. Anyway, this would break backward compatibility for the whole documentation.

I realize that these would break the websites "go to the current version" feature...how critical is that?

I don't know. This wouldn't be the first library where this was done. I face this issue quite often with other libraries. It's simply annoying but I'm not totally against it.

I simply doubt that all of the above is really needed in practice, AFAIU you only have problems with the paths containing these long filenames, correct? So these ones for sure:

rtree_iterator__iterator__parameters_type_const____indexable_getter_const____value_equal_const____allocator_type_const____packalloc_const___.html
rtree_parameters_type_const____indexable_getter_const____value_equal_const____allocator_type_const___.html
rtree_range_const____parameters_type_const____indexable_getter_const____value_equal_const____allocator_type_const____packalloc_const___.html

and these maybe, but not really:

rtree_iterator__iterator__allocator_type_const___.html
rtree_iterator__iterator__allocator_type_const____packalloc_const___.html
rtree_range_const____allocator_type_const___.html
rtree_range_const____allocator_type_const____packalloc_const___.html
rtree_rtree_____allocator_type_const___.html
rtree_rtree_const____allocator_type_const___.html

correct?

[teeks99]

Yeah, fixing those would correct my issue and preserve backwards compatibility.

Beyond that it comes to generally supporting boost in deeper directory structures vs. backwards compatibility.

[mloskot]

@awulkiew Thanks for the fix!