Should we continue development on dev branch

[huchenlei]

@lllyasviel has been inactive in maintaining forge for about 2 months. Collaborators do not have permission to merge to main now. Should we continue our development in dev and let all our users switch to dev branch instead?

[huchenlei]

I think another option is that we try to merge SD Forge's memory management back to main A1111 repo. I think this might be doable if we can setup good enough memory profiling for torch, and pinpoint the root performance issue in A1111.

[strawberrymelonpanda]

I think another option is that we try to merge SD Forge's memory management back to main A1111 repo

Has anyone talked with @AUTOMATIC1111 (via Discord?) for his opinion on the Forge backend? Upstreaming these changes would make life easier I feel, but whether any attempt would ultimately be merged really depends on if he's on board.

[catboxanon]

@strawberrymelonpanda Yes. Auto's answer was they have no consideration in "using comfy in any way" (ultimately where Forge's memory management and ldm module comes from). Source (from the A1111 dev Discord).

This discussion that took place on Discord in August 2023 is the reason this fork exists to begin with.

[strawberrymelonpanda]

Thanks for the info! (I'm not a discord member and wish discussion had never moved there, but oh well.)

So yeah, sounds like it would come down to improving his / the CompVis(?) implementation. That'd be wonderful, but I have to admit I'm skeptical after all this time how practical. Godspeed. 👍

[slashedstar]

I mean, everything is reversible anyway; there's no reason not to do it, as is the project is just rotting away for (seemingly) no reason.

[strawberrymelonpanda]

I'm on board and voted yes, but sadly I think the biggest reason would be lack of visibility. Most people aren't going to know to look at "dev2" and as things stand they can't be merged back into Main without @lllyasviel. Even the ReadMe can't be changed.

I hate to suggest what would be basically a fork of a fork, but...

[huchenlei]

I have merged some trivial fix to https://github.com/lllyasviel/stable-diffusion-webui-forge/tree/dev2. Please switch there and try out these patches. Thanks!

[TheRealPhooney]

If there had been any indication from @lllyasviel himself that he would ever do anything on it again, I'd vote to wait, but since that isn't the case it feels like a good idea to do something to move forward and I appreciate you stepping up.

Let's face it, a single comment saying he got busy with real life or another project and would or would not be back to continue development could have been done if he wanted to make it assuming he still had internet access, which he did.

It's a shame, but he owes us nothing, this is just the downside of using a product/service/demo at the cutting edge like this. The upside is that the code is available for others who can grasp it to continue development, taking the wonderful work that @lllyasviel did and running with it.

[E2GO]

It's may be some private own projects or something cooking for future for everybody. Check out @lllyasviel activity on private repos. Anyway dev2 branch is a good decision meanwhile and let's have some patience - may be we'll be great surprised by @lllyasviel as usual with his team's projects)

[altoiddealer]

This is pretty interesting timing... great minds think alike?

Check out this Discussion I opened on Auto1111 just last night. No response yet but,...yeah, the inactivity in Forge development is starting to get unnerving, knowing that the next best alternative comes with a significant expense to performance and memory.

The community will be truly blessed if another motivated dev (such as yourself, it seems!) were to save this from extinction.

[LadyFlames]

#680 if you check this Thread apparently someone got into contact with illya indirectly through discord just scroll to the bottom of that thread

[gohan2091]

Can someone show me how to update my Forge to the new Dev2 version?

[gohan2091]

Why would anyone be using dev2 then if it completely breaks ControlNet? I have no function available in ControlNet, every option results in the same error. Dev2 adds Multi-Input tab but why add that if the entire ControlNet is broken? Posted it in the issue section here: #709

[CCpt5]

Yea, it's all fluid - I don't know.

To roll back just do the opposite and git checkout main, pull, and should be as you had it.

[gohan2091]

I don't mean to sound ungrateful but the main branch lacks Multi-inputs tab and the dev2 adds it... great.... except now the entire ControlNet in dev2 is broken so why add the tab to a broken feature? makes no sense. Yes I have rolled back to the main branch for now so I can use ControlNet again. Thanks for your help.

[CCpt5]

Yea,, that's kinda the hope of people in this thread - that updates continue....there's a bunch of stuff that's outdated and bugs that have cropped up....best case is the main developer is lllyasviel ok and comes back to the project. 2nd best case is that huchenlei continues on dev2 or elsewhere (fork?). Until then A1111 is the alternative (or can use in tandem).

[gohan2091]

I use to use SDNext as my main but that's become such a mess in my opinion although it is faster. I will continue on with the main branch of Forge for now. Let's hope...

[thiagojramos]

If you can continue this, even with a possible return of illv on the horizon, we will be eternally grateful. I tried to make some changes by myself and... well... I had to revert everything back, haha...

[GavChap]

Should I do a pr into dev2 for the same commit as I've done for main then?

I've got another one waiting in my fork for the Align Your Steps scheduler as well.

[wywywywy]

Align Your Steps scheduler

That'd be amazing!

[freecoderwaifu]

It is curious what lllyasviel might be cooking in his private commits, and at least he seems to be ok if he's active behind the scenes.

That said, I think the show must still go on in some way to avoid falling behind, and the dev2 branch is a good compromise.

[devils-shadow]

I've been refreshing the Forge git page randomly for the past few months, waiting for updates. I never once assumed the project was dead because as it stands, at least for my use-cases, it is rock solid, with top-notch performance and no bugs that have affected my day-to-day usage.

I just assumed that's the reason for a lack of updates, because adding random stuff might lead to performance degradation.
I'd like to believe that llyasviel will return when he has the time. Until then, for anyone wanting to experience a more cutting-edge build, dev2 is the way to go.

I really hope Forge continues as a project. SD.Next is excellent but simply refuses to allow me to do txt2img upscales due to OOM (diffusers/SDXL), and automatic1111 is slow. Forge has been the best thing to happen to my workflow since I discovered it.

[MNeMoNiCuZ]

There are a bunch of quite critical performance updates to the Extra Networks tab that are being worked on for A1111. Once these are fixed, I may switch back to vanilla A1111 just to be able to choose models more effectively when working.

AUTOMATIC1111/stable-diffusion-webui#15530

I may work in A1111 when setting up prompts, and then switching to Forge for speeds when generating for a long duration.

[gohan2091]

I've been refreshing the Forge git page randomly for the past few months, waiting for updates. I never once assumed the project was dead because as it stands, at least for my use-cases, it is rock solid, with top-notch performance and no bugs that have affected my day-to-day usage.

I just assumed that's the reason for a lack of updates, because adding random stuff might lead to performance degradation. I'd like to believe that llyasviel will return when he has the time. Until then, for anyone wanting to experience a more cutting-edge build, dev2 is the way to go.

I really hope Forge continues as a project. SD.Next is excellent but simply refuses to allow me to do txt2img upscales due to OOM (diffusers/SDXL), and automatic1111 is slow. Forge has been the best thing to happen to my workflow since I discovered it.

Is your ControlNet working in dev2? because for me it's completely broken

[thiagojramos]

I haven't tested ControlNet after switching to dev2, but after reading about your experience of finding it broken, I went to test it and indeed, it's also giving me errors.

[gohan2091]

Thanks for confirming. I just don't understand why the Multi-input tab was added to ControlNet when the entire ControlNet is broken. Kind of pointless. Hope it can be fixed sometime :)

[thiagojramos]

I believe the update (to add multi-input) broke the integrated ControlNet in Forge (as far as I remember, the one in main works fine). It's just a guess, but I believe the updated code works for the case of the normal ControlNet (the external extension), but since the integrated one is a bit different, maybe that's the issue.

[Moussack]

is @lllyasviel still working on IC-Light ??
hope he's not abandoning Forge.

[LadyFlames]

as far as anyone knows he is working on his Private Projects those on his Discord Server would probably know more on what he is doing

[gohan2091]

I get that Illyasviel has given his time for free and we should (and are) grateful for Forge but no communication from him? no quick announcement to let us know the current situation? Get our hype up with this UI and then abandon us for an unknown amount of time. Very disappointing that Illyasviel can't spend a couple of minutes writing us a quick message.

[thiagojramos]

Just to reiterate, he abandoned the project. I'll use the example of Fooocus, which I closely followed. Last year, when he needed to step away to focus on other projects, he created a post letting everyone know and mentioned he would only merge important PRs from the community. Even now, he left someone in charge of Fooocus and used to respond to people’s posts whenever he could.

But with this project, after that controversial discussion about using something from ComfyUI in Forge (I’m not exactly sure about the details), he just vanished.

To me, it always seemed like this project was more of an experiment for him. Maybe, after that discussion, he decided to drop it for some reason. Unlike Fooocus, which is his "legitimate child," Forge seems to be the "problematic adopted child." Sorry for the metaphor, but that’s what came to mind.

In short, I think he gave up on this project for the reasons I mentioned.

[itsuxX]

Hello everyone, can someone please explain how to switch to DEV2? Where do I need to configure something? I'm just a user. Thank you very much.

[thiagojramos]

Here: https://gprivate.com/6bepe

Break (click) in case of emergency

First, run git switch dev2 followed by git pull. If you encounter the error "Your local changes to the following files would be overwritten by merge", type git stash. If the error persists, reset everything with git reset --hard and then try git pull again. That should solve the issue.

Note: Make a backup of your webui-user.bat before using git reset --hard because it will reset the file. You can replace the old file with the new one afterward.

[itsuxX]

Thank you very much

[VitaliyBd]

I'm not a developer, and I could be completely wrong, so please pardon me if I am. Doesn't it make more sense to integrate Forge memory management in A1111 instead? To my understanding, multidiffusion and Tiled Vae already work in a similar manner.

[thiagojramos]

#166

[amitsandhel]

So i'm a bit confused as I too want to help develop this or would like to spend time but not sure if it would be helpful to the community ?
so is forge kind of dying and the main developer ended up building fooocus which is the one he is investing his time on ?
Also seems we can't take the speed advantage forge has and put into into A1111 as those developers don't want to do that for some odd reason ?
I mean is the backend of forge so different that nothing can be even ported to a1111 ?

[amitsandhel]

hmm I'll test out that Pr but if that PR does get added would forge have any use case left then ?
I want to actually develop and code some stuff for this and learn how to do it but I would like to do it in a project which won't be "dead" so to speak or won't go anywhere

[valery016]

#15821 sounds like a great news! So there is a hope to lift Automatic to Forge's level of speed, after all. What about memory management, another Automatic's week spot, is it being addressed in #15821? Also, I am afraid we are forgetting here the main issue with Automatic, next to its poor speed: it's a messy piece of software. Every now and then I run into a crash, or a cryptic error message, or just a paralysis of Automatic, when using it with high resolution images. If someone really is determined to straighten out Automatic, this should become a priority as well. If Automatic continues being such a mess, I will look elsewhere, where bugs are still being fixed (if not Forge, then SD.Next, StableSwarm, and even Comfy).

[amitsandhel]

#15821 sounds like a great news! So there is a hope to lift Automatic to Forge's level of speed, after all. What about memory management, another Automatic's week spot, is it being addressed in #15821? Also, I am afraid we are forgetting here the main issue with Automatic, next to its poor speed: it's a messy piece of software. Every now and then I run into a crash, or a cryptic error message, or just a paralysis of Automatic, when using it with high resolution images. If someone really is determined to straighten out Automatic, this should become a priority as well. If Automatic continues being such a mess, I will look elsewhere, where bugs are still being fixed (if not Forge, then SD.Next, StableSwarm, and even Comfy).

Here is some of the testing I did. I have a Nvidia GTX1050Ti Graphic card with only 4Gb of VRAM. i don't have a lot of VRAM and I am on Linux using PopOs a flavor of Ubuntu. I was trying to run SDXL checkpoints and generate images with that.

• SdNExt: I tried sdnext there using a different backend called diffusers. They are fast Currently there getting a lot of errors since they broke off from A1111 and now are a new thing. For me sdnext doesn't work. With respect to SDXL it crashes it can generate a basic image but I can't use face restore and even highres fix since I go out of memory. The same is true with using SD1.5 with a 512x768 imge I couldn't use their highres fix or diffusion fix and the images were different from what civitai or A1111 or forge generates. I ended up uninstalling and deleteing SDNext sadly.

• Comfy: I did try comfy last night. It was easy to use actually and ironically I did see that the main time spent by the process was in the sampler where it was using the sampler to get an image out the door. For me using euler A for 24 steps on my nvidia gtx1050Ti card with 4Gb of Vram it was doing about 36 sec/iteration. Do note this was for a SDXL checkpoint an aaaautism checkpoint and I was generating a simple anime girl picture and pose. I did run comfy with the --lowvram command but saw no real change or impact in performance. i am not using controlnet or anything for this test. Just 3 loras and a basic sampler and checkpoint. i ended up uninstalling this too as it was too slow.

• Forge: I have tried forge and use forge to this day. Ironically with there OOm and freeU and even with their optimization I can run SD1.5 checkpoints and models at a a speed of 1-2sec/iterations. With respect to controlnet I can even use controlnet on a SD1.5 checkpoint and model easily with little speed loss it becomes about 8 seconds/iteration.
  - Furthermore I can even run SDXL/pony checkpoints at about 24sec/iteration with forge. I typically use Euler A with a step count of 24. I can even use Adetailer as well to fix the faces.

One new thing I discovered are the sdxl lightning checkpoints which by using the LCM sampler you can generate an image in 9 steps which is an interesting approach it takes 24 second but only gotta suffer for 4-9 rounds.

I have not tried a111 yet but the first time i ever installed it was months ago around january of 2024 and it was slow like 40sec/iteration but I can't say anything conclusive about that.

So far it seems forge is the best for low graphic card users or at least the best in terms of performance. So it could be worth the time to invest in dev2 and build new features or possibly think of new ways to get more bang for our buck.

Hope this helps.

[Moussack]

#15821 sounds like a great news! So there is a hope to lift Automatic to Forge's level of speed, after all. What about memory management, another Automatic's week spot, is it being addressed in #15821? Also, I am afraid we are forgetting here the main issue with Automatic, next to its poor speed: it's a messy piece of software. Every now and then I run into a crash, or a cryptic error message, or just a paralysis of Automatic, when using it with high resolution images. If someone really is determined to straighten out Automatic, this should become a priority as well. If Automatic continues being such a mess, I will look elsewhere, where bugs are still being fixed (if not Forge, then SD.Next, StableSwarm, and even Comfy).

Here is some of the testing I did. I have a Nvidia GTX1050Ti Graphic card with only 4Gb of VRAM. i don't have a lot of VRAM and I am on Linux using PopOs a flavor of Ubuntu. I was trying to run SDXL checkpoints and generate images with that.

• SdNExt: I tried sdnext there using a different backend called diffusers. They are fast Currently there getting a lot of errors since they broke off from A1111 and now are a new thing. For me sdnext doesn't work. With respect to SDXL it crashes it can generate a basic image but I can't use face restore and even highres fix since I go out of memory. The same is true with using SD1.5 with a 512x768 imge I couldn't use their highres fix or diffusion fix and the images were different from what civitai or A1111 or forge generates. I ended up uninstalling and deleteing SDNext sadly.
• Comfy: I did try comfy last night. It was easy to use actually and ironically I did see that the main time spent by the process was in the sampler where it was using the sampler to get an image out the door. For me using euler A for 24 steps on my nvidia gtx1050Ti card with 4Gb of Vram it was doing about 36 sec/iteration. Do note this was for a SDXL checkpoint an aaaautism checkpoint and I was generating a simple anime girl picture and pose. I did run comfy with the --lowvram command but saw no real change or impact in performance. i am not using controlnet or anything for this test. Just 3 loras and a basic sampler and checkpoint. i ended up uninstalling this too as it was too slow.
• Forge: I have tried forge and use forge to this day. Ironically with there OOm and freeU and even with their optimization I can run SD1.5 checkpoints and models at a a speed of 1-2sec/iterations. With respect to controlnet I can even use controlnet on a SD1.5 checkpoint and model easily with little speed loss it becomes about 8 seconds/iteration.
  • Furthermore I can even run SDXL/pony checkpoints at about 24sec/iteration with forge. I typically use Euler A with a step count of 24. I can even use Adetailer as well to fix the faces.

One new thing I discovered are the sdxl lightning checkpoints which by using the LCM sampler you can generate an image in 9 steps which is an interesting approach it takes 24 second but only gotta suffer for 4-9 rounds.

I have not tried a111 yet but the first time i ever installed it was months ago around january of 2024 and it was slow like 40sec/iteration but I can't say anything conclusive about that.

So far it seems forge is the best for low graphic card users or at least the best in terms of performance. So it could be worth the time to invest in dev2 and build new features or possibly think of new ways to get more bang for our buck.

Hope this helps.

Yeah forge is excellent for low end hardware, I've tested it with my gtx 970 4GB i can generate 3215x2200 img below with hiresfix with reasonable amount of time (7 minutes). It's impossible to do that in A1111

[BNP1111]

can i ask how to do so great img,thank you

[aolko]

Can you please update controlnet? Interested due to composition ip-adapter

[amitsandhel]

Can you please update controlnet? Interested due to composition ip-adapter

That's the question how no one here has access to the permissions to push to master. We would need some way to organize ourselves to work on this assuming there is worth in doing so and this is doing something both a1111 and comfyui can't do?

[aolko]

Can you please update controlnet? Interested due to composition ip-adapter

That's the question how no one here has access to the permissions to push to master. We would need some way to organize ourselves to work on this assuming there is worth in doing so and this is doing something both a1111 and comfyui can't do?

dev2 branch, come on

[amitsandhel]

So is A1111 still slow performance wise after all this time and is anyone using comfy and knows what speeds their getting from say a weak graphic card
maybe we can do some speed comparisons to see how forge is still faring ?

[VitaliyBd]

IMHO, stay away from comfy. It's buggy, poorly written software by someone who has no idea what they're doing (node-based UI without built-in custom node management). It's slowly being abandoned by the community. You'll spend a large amount of time just setting up your render, whereas it only takes one or two clicks in A1111 with much better results.

[aolko]

however a lot of people port shit to nodes from papers, no such love for a1111/forge :(

[VitaliyBd]

I think that's because when custom node authors abandon their projects (and 90% of them already have, especially the most promising ones), this "paper" feature stops working after updates and causes many problems for other nodes, making this feature absolutely unusable.

[amitsandhel]

I tried A1111 today sadly it just didn't work on my 4gb of vram it crashed my laptop instead forcing me to do a force hard shutdown and restart lol

Forge is still better at least both sdxl and 1.5 work now if only we can figure out which feature and improvements we wish to add and have maybe we can get coding

[aolko]

I tried A1111 today sadly it just didn't work on my 4gb of vram it crashed my laptop instead forcing me to do a force hard shutdown and restart lol

Forge is still better at least both sdxl and 1.5 work now if only we can figure out which feature and improvements we wish to add and have maybe we can get coding

How about stopping ui lag with extensions installed? #716 (comment)

[johiny]

SD forge is amazing lllyasviel did an incredible work on memory management, I don't know what kind of memory management A1111 has but it literally crashed my system on Linux with ROCM.
I have tested SD Next, Comfy UI, A1111 and all has some kind of weird memory behaviors, like crash the whole system or eating up so many resources I guess majority vram? that secondary screens become black.
But Forge has given me the most stable experience, and for me after changing to AMD and passing through the literal hell to make SD work.
I really appreciate that stable behavior, so I think work on dev branch meanwhile lllyasviel comes back or not is a nice idea.

[CantainPike]

You may try StableSwarm still in beta which is works faster than on my RTX 3060 12GB VRAM, Forge is one image generation for SDXL 17.3 seceond, StableSwarm 15.6 Second Slightly faster. I wonder how does it works on low VRAM machines. Keep in mind StableSwarm still in beta some features are under developement.

[amitsandhel]

hmm but stableswarm is built In C# as I looked at the codebase uses dockers too so I don't know if that would be useful since dockers from my beginner's experience eats up lots of disk space and you need dotnet8 as well

[aolko]

it's ugly AF and has a bad UX

[altoiddealer]

it's ugly AF and has a bad UX

Been lurking this thread, but just want to pop in an mention that the author of StableSwarm (or at least the key dev / face of the project) is esteemed community member McMonkey. Besides, that comes across a bit disrespectful for anything open source, freely gifted to us

[aolko]

it's ugly AF and has a bad UX

Been lurking this thread, but just want to pop in an mention that the author of StableSwarm (or at least the key dev / face of the project) is esteemed community member McMonkey. Besides, that comes across a bit disrespectful for anything open source, freely gifted to us

what truly is disrespectful is denying user contributions

[aolko]

oh and remind me who abandoned stablestudio ui real quick

[amitsandhel]

It seems after some research we have conducted we (the small group that interacted in this discussion post) could conclude forge has value and we should code and work on it.

It still has the fastest speed and works beautifully on low graphic cards and even high end cards can generate higher quality images that they can't do otherwise using other tools.

So then we have two questions

1. do we have any tutorials to read on how to code or write some code for this
2. who wishes to manage this branch or do we just fork it and make our own branches and make pr's for it. I doubt anyone should be self inserting their own code into dev2 ?
3. do we create some more appropiate communication stream like sdnext did using discord or something ?
4. anyone got in contact with the main author and asked him about forge ?

[aolko]

let's fork this and make a discord server for it

[amitsandhel]

Sounds like a plan let's get cracking
Autobots roll out !
What's the discord server btw ?

[PanduZen]

@huchenlei

Now that the announcement has happened, are you planning to keep updating the dev2 branch?, I don't mean to implement new complex things, but to fix bugs and update basic things for those who want to keep using forge?

[obs945]

@huchenlei

Now that the announcement has happened, are you planning to keep updating the dev2 branch?, I don't mean to implement new complex things, but to fix bugs and update basic things for those who want to keep using forge?

Can someone make a simple tutorial to help us switch from main to dev2 branch ?
I tried "git checkout dev2 && git pull" but in vain. The process was terminated during modules/launch_utils.py modification.

[PanduZen]

@huchenlei
Now that the announcement has happened, are you planning to keep updating the dev2 branch?, I don't mean to implement new complex things, but to fix bugs and update basic things for those who want to keep using forge?

Can someone make a simple tutorial to help us switch from main to dev2 branch ? I tried "git checkout dev2 && git pull" but in vain. The process was terminated during modules/launch_utils.py modification.

use stability matrix

[hoodady]

Now here's how to switch, nice and easy:

Go to the root directory of your Forge installation (i.e. whichever folder has "webui-user.bat" in it)

Open a terminal window inside this directory

git pull (updates Forge if it isn't already)

git fetch origin (fetches all branches)

git switch -c dev2 origin/dev2 (switches to the dev2 branch)

Done!

[hoodady]

If you'd ever like to switch back, just run git switch main from the terminal inside the same directory

[MNeMoNiCuZ]

Let's go! Forge2 <3

[aolko]

Wait, that's not "resolved", is it?

[altoiddealer]

Perhaps Ill has now returned to save us all.

(I won’t hold my breath)

[aolko]

Perhaps Ill has now returned to save us all.

(I won’t hold my breath)

nope

[prog-ape]

@huchenlei Hello, will there be further updates? I'm eagerly looking forward to them. Even though I have seen your patches in the web UI, the execution speed during API calls in the web UI is still very slow compared to Forge.

[aolko]

here's one:
move to https://github.com/Panchovix/stable-diffusion-webui-reForge