Importing model from nested structure. about gae-init

[mdxs]

Should you not use the import .model approach (instead of the from server import model) inside the main/server/model/user.py? You may also want to have a look at the __init__.py in your model folder, I guess it does that and the way it works, this may inspire you to understand what to add in your situation (I needed to add some imports myself to get the additional stuff in). HTH

[joernhees]

@topless out of the blue: did you try a from __future__ import absolute_import at the top of the files? otherwise the old behavior is to check the current package first, maybe you have a name clash there?

[topless]

@mdxs import .models still gives the error. The dot notation is only used in the __init__.py file.

@joernhees All my model files start with absolute_import and the __init__.py is in the same fashion like gae-init from .base import Base and so on..

I have been going through every line of change very thoroughly since yesterday..

[joernhees]

do you have both or just one model?:

/main/model/...
/main/server/model/...

[topless]

Only the second one.

main/server/controllers
main/server/model
main/server/templates
main/server/lib
main/server/main.py
..
main/static

is how the top level of my structure looks like.

[joernhees]

and main/server/model/__init__.py imports all the modules in that package?
(sorry for maybe obvious questions, remote debugging kinda sucks)

[mdxs]

@topless consider a cut-down example on GitHub: no real work, just a gae-init fork that you modify to resemble your changes enough to reproduce the issue. You might just find the issue doing so...

[topless]

@joernhees don't be, I am grateful for your help guys.

My __init__.py looks like

from .base import Base
from .config import Config
from .config_analysis import ConfigAnalysis
from .user import User
from .user_beta import UserBeta
from .study import Study
...

@mdxs I will port my changes in a new gae-init project to cut it to bare minimal.

[joernhees]

@topless so if the line from server import model fails in main/server/model/User.py, can you above that insert these:

import sys
import logging
logging.warning(sys.path)

also i assume you have a main/server/__init__.py as it is working in config.py...

[lipis]

You can also provide the actual error message from the console, maybe will give us more clues..

also remove the try/catch from the main/config.py until everything will be solved, because that try/catch is needed for run.py and for the time being run the dev server manually.. like the good old times :)

[joernhees]

that try/catch shouldn't hide problems anymore i hope ^^

[joernhees]

@topless and one more thing: you might run into a circular import lock... can you check that the order of imports in all files in which you import models is the same?

(this is a problem arising from the criss-cross import from config.py of the Config ndb.Model, you should be fine if you keep the ones from gae-init the way they were and your own below it, but if you swap the order in main/server/model/__init__.py and one of your own modules in which you essentially import that __init__.py again by from server import model, then your own modules can create a deadlock on themselves)

[topless]

@mdxs Minimum changes architecture.
hg clone https://[email protected]/topless/gae-init-nested for pasting glory.

This is a simplified version of what my project looks like.

@joernhees I will have a look at it. Although I doubt that they have a consistent order.

[topless]

Quoting (with minor modification) the comment from this If model's parent directory is not in sys.path then the presence of __init__.py in the model directory doesn't help much This is actually the reason I tried to add my server to appengine_config.py.

[joernhees]

ugh, modifying the path is always ugly...

[joernhees]

hmm, the more i play around with this, the more i think this is one of the shortcomings in the python import system clashing with our code guideline of importing the module...

if rather than from server import model you do from server.model import Base everything seems to work.

debugging the first version and server it shows that at the time this is called server doesn't seem to have a name model yet, as we're still in the loop of defining it...:

Traceback (most recent call last):
... bla
  File "/gae-init-nested/main/server/main.py", line 8, in <module>
    import config
  File "/gae-init-nested/main/config.py", line 27, in <module>
    from server import model

##### this is where we first define server.model:

  File "/gae-init-nested/main/server/model/__init__.py", line 4, in <module>
    from .config import Config

##### this is the second loop where we load server and didn't finish the above:

  File "/gae-init-nested/main/server/model/config.py", line 17, in <module>
    from server import model
ImportError: cannot import name model

A import server ; logging.warning(dir(server)) shows: ['__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', 'controllers'], no 'model'.

I guess the reason why the second one works and the first one doesn't has to do with the from <module> import <anything> syntax: if server.model is in the <module> spec part, python can assume it is a module safely and resolve the loop by assuming "haha, yes, that will still come, no worries, it's a module and i know it as we're just loading it", but as soon as it is in the <anything> part this can't be done safely anymore as it could be a variable, function, class, ... anything :-/

So i guess the proper solution is to actually use from server.model import Base or easier: from .base import Base

[topless]

Are you aware of any particular reason that the submodule is still loading in the nested thing, while in the bare gae-init works properly?

[joernhees]

good question... not really. only difference coming to my mind is that in gae-init it is a top-level package and here it isn't

[mdxs]

@topless Having no idea whether it makes any difference, but running your gae-init-nested locally I find myself with a main/lib/ and main/server/lib/ folder with seemingly same content. This is after ./run.py -C and ./run.py -s. Expected/required?

[mdxs]

Hmm, when I change in main/server/model/config.py the import model into 'import server.model'; then I get one step further... then it fails on import util (which makes sense, as you pushed it into server/controllers/

[mdxs]

Sorry for my rambling... I should investigate more before commenting; please ignore above 2.

[gmist]

https://bitbucket.org/topless/gae-init-nested/pull-request/1/fix-imports/diff

[mdxs]

@topless See https://bitbucket.org/mdxs/gae-init-nested/commits/adc9f831c85c4defd16c1c82fd206b5e82ad88a4 for some changes that made it work locally for me

[mdxs]

Probably the same as @gmist (but his approach is cleaner...)

[lipis]

So @topless did you manage to finally running the app after the merge?