Dynamic project structure

[liorbenhorin]

Idea:

Eliminate the category - asset and the sequence - shot structures.
Instead allow the user to create folders and nest components inside them.
He can just store them in the root of course.

Option to create some default folders ( assets, animation, lightning, etc )
Option to filter folders to recreate the scenes and assets mode idea.
For ex. show only the contents of "Animation" folder.

Legacy support:

Adding a pipeline version tag on the project.pipe file
If no tag is found that means the project is old.
Create a module that converts categories, assets, shots and sequences into folders in the main module.

Extensive change. well worth it.

[RobotSnowOtter]

To clarify what you mentioned earlier in your e-mail. If someone opens explorer do they see every single maya file for the whole project? If so, I think there's some value to keeping it a little bit contained and navigable, but only to a certain point. I looked at my thesis and the project itself had 1800 .ma files. While the main character asset, which was the most complicated had 234.

I think there's some value to being able to still navigate it in explorer. I'd fear that if something went wrong with the .pipe file, I'd have no way to make sense of those files if they were all in the same place. If there's subgroups within Assets and Scenes, I think that's a good division. For example, if assets and scenes each had 3 subfolders, those 1800 .ma files would average to 300 per subfolder which seems like a good balance. It would also safeguard the other 5 if one .pipe file did become corrupt.

Only if a project had less than 18 .ma's would you end up with more folders in a directory than files (which is when you start to feel like you have to click too much, but even then, it's less than it is currently). And I think on the other end of the spectrum, a project could be substantially bigger before you started having too many subfolders (within assets or scenes) that it got congested.

[liorbenhorin]

@show621
Sorry iv'e missed your comment...
You are correct.

Here is a description of the system i am thinking of:

GUI side:

project tree:

[-] component (eg. Something very simple that dose not made of separated elements)
[-] asset (eg. Superman)
 |---- components (eg. Model, Rig )
[-] folder (Optional, for sorting purposes, this was the role of 'catagories')
 |---- [-] assets
         |---- components

Component is made from a master file, versions and master versions

Behind the scenes:

'folders' are virtual and won't exists on hard drive
project assets folder will look like this:

project/assets/asset/component/asset_component_MASTER.ma
project/assets/asset/component/versions/asset_component_##.ma
project/assets/asset/component/masters/asset_component_MASTER_##.ma

Or - if a component is not nested inside an asset:

project/assets/component/component_MASTER.ma
etc...

What we can do with it:

folders:

1. use for sorting purposes, if you want too.
2. rename and move with no effect on ref links.

assets:

1. move in and out of folders all the time and have no effect on ref links.
2. rename breaks ref links

components:

1. move between different assets or folders, but this will break ref links.
2. rename breaks ref links.
3. hopefully ability to duplicate

This architecture can take me a while to create, but it will be worth it.
I have spent the last week trying to figure out how to get a tree view and a data model to work - It sounds so simple but it was very hard.

Here is a mockup of the idea:

Give this some thinking and let me know!

[RobotSnowOtter]

A concern I have with the outliner layout (and the previous 3 column layout) is that it isn't very different from a file browser. When I previously mentioned the relationship of the outliner and display layers, I meant it abstractly. Most file browsers are setup similar to the outliner tree, but a pipeline interface could simplify that and be for flexible like display layers. I also wanted to point out that that the outliner layout is very vertically oriented and risks taking up a lot of your GUI's real estate or cause excessive scrolling on larger projects (which could be avoided otherwise). Have you checked out simpleTools? They have a very compact but usable tab and sub-tab layout (though, I think it goes too in depth after the asset/shot level).

I certainly appreciate that you can have a simple component without having it tucked into an asset. One quick note, in terms of how we're defining things; "components" always seemed strange to me in the days of open pipeline, but in your component tree example it seems even more strange when arranged that way and I was a little confused at first. Say I have a simple asset - like a cup or a leaf. Those would both be considered components and only something like a character or a vehicle or maybe a set would be considered an asset. To me it makes more sense if simple things like the cup or leaf are still considered "assets" and more complex things like vehicles and characters are considered "asset collections" or maybe "compound assets" or an "assembly" - something like that, instead of "components" and "assets." I only mention it because I think it helps clarify it's purpose and function. However, the definition might be a moot point if you agree with my remaining note, which is that the component level doesn't need to exist outside of virtual folders.

Personally, I would rather have simplification and virtual folders happening at the micro, component level, rather than the macro, category level.

For example in the old system you might have had something like:

assets\

• characters\
  • superman\
    • model\
    • rig\
    • etc\
  • [other characters]
• props\
  • table\
    • model\
    • texture\
  • [other props]
• sets\
  • alley\
    • model\
    • debris\
    • etc\
  • [other sets]

17 directories, 3 levels deep

As I understand it, what you're describing would look like:

assets\

• superman\
  • model\
  • rig\
  • etc\
• [other characters]
• table\
  • model\
  • texture\
• [other props]
• alley\
  • model\
  • debris\
  • etc\
• [other sets]

14 directories, 2 levels deep

...at least in the file browser. In the pipeline interface, it would be up to the user to categorize them to keep it from getting too big. But in the file browser all the assets, simple and complex, would share the same folder space.

What I had in mind would look like:

assets\

• characters
  • superman
  • [other characters]
• props
  • table
  • [other props]
• sets
  • alley
  • [other sets]

9 directories, 2 levels deep

To me, that is about as simple as it's going to get while staying easy to navigate simple and complex assets. Hopefully those examples demonstrate that visually. As we discussed, a potential problem with this approach is with complex assets like "superman" which may have hundreds of files and dozens of components in a single directory. However, ideally, that's where the virtual [sub] folders (or tags) would come in.

So in pipeline when you have superman selected you could create a "rig" sub-group. Maybe after working on rig for a while, you realize in the last 5 files you've been working on the facial rig and it would be good to create another sub group called "facial rig" and move those there. They'd still be in the superman directory, but in pipeline, they'd have their own space. It would be amazing if the sub-groups affected the file name. I think then, even with hundreds of files in a directory, you could clearly navigate in the file browser if you needed to. I believe it's really valuable to be able to leverage the OS file browser and not solely rely on the interface (like many other tools force by creating overly complex directory structures). I'd hope that the above setup would steer users to operate in a simple, but organized way, but then give them the freedom and flexibility on a micro/component level to group things (or not) however they see fit.

[liorbenhorin]

Hi!!
Sorry for the delay,
I started a new project and can't find time for anything :/

I am currently building a pipeline for a large scale studio here in Tel Aviv, and getting some nice ideas for Pipeline.

I've got much to say about your note, but i need time to write it down :)

Talk soon!