nerdctl commit taking long time possibly because of fs.Changes walking through all files

[pankajkumar229]

We have an 8GB image we are trying to commit with nerdctl. It takes a long time for this to happen. It seems to be because fs.Changes implemented in continuity is not using the diff functionality from overlayfs but just walking every file.

continuity/fs/diff.go

Line 101 in 206f576

func Changes(ctx context.Context, a, b string, changeFn ChangeFunc) error {

It seems like this needs to be implemented here:

https://github.com/containerd/continuity/blob/main/fs/diff_unix.go#L34

Could someone please take a look to see if our understanding is correct. If it is, what would it take to fix this code so it can work more effectively for overlayfs?

[AkihiroSuda]

Link:

• fs: add DiffDirChanges function to get changeset fast #145

[pankajkumar229]

Thanks for pointing this out. The changes look like they are almost complete. Wondering why this was never merged. if it would be possible to make this fix work by people unfamiliar with this codebase in a reasonable amount of time.

[guyLee687]

@AkihiroSuda Do you know if this issue can be closed and when #236 (comment) will be deployed to production for containerd?

I'm also trying create a service using "nerctl commit" without the large latency involved.

[gogasca]

@pankajkumar229 @AkihiroSuda any status on this? we are interested in seeing this solved and would be willing to contribute to fix.

[AkihiroSuda]

#145 has been merged but probably not used in nerdctl yet.
Investigation and contribution is highly welcome.