Testably vs. TestableIO?

[rwv37]

It seems like these two projects are somehow related (I don't mean necessarily in the sense of their code). TestableIO.System.IO.Abstractions's GitHub site mentions Testably.Abstractions as a "related project" that "provides alternative test helpers and additional abstractions", and it looks to me like at least one main contributor here is also a main contributor there.

Are they intended to be usable together in some complementary way? Is one intended as a replacement for the other? When would I want to use one rather than the other?

For context: I've been using TestableIO for a few weeks, and have not used Testably at all.

Thanks in advance.

[vbreuss]

Yes, they are indeed related.

Testably uses the interfaces from TestableIO.System.IO.Abstractions so that no changes on the productive code is necessary, when switching the testing library. The testing helpers in this project are much more extensive (e.g. time handling, support for drives, the file system watcher or working with SafeFileHandles is possible here, but not in System.IO.Abstractions) and also include further interfaces (ITimeSystem and IRandomSystem).

I am currently the main contributor in both projects, but my focus is definitely on the Testably-projects.
If you want to switch, you have to reference Testably.Abstractions.Testing in your test project and use the MockFileSystem from there. You don't have to change anything in your productive code!

[rwv37]

Thanks. I've tried this out and have been confused by the results. Please forgive me if I am misunderstanding, but I now believe the source of my confusion is your usage of the word "productive" in "You don't have to change anything in your productive code". You've used the word in this confusing-at-least-to-me way on the project's documentation page, too, so I thought it would be best to mention it to you so that you could fix it there:

I now think you don't really mean "productive code", and instead mean "production code", i.e. the "real", non-testing code. At least to me "productive code" doesn't mean that. Instead, it means something like "functioning" or "working" or "useful". So, when I replaced my productive test project's productive code's references to TestableIO.System.IO.Abstractions.TestingHelpers.MockFileSystem with references to Testably.Abstractions.Testing, I was surprised to get a whole bunch of compilation errors.

I see that you seem to have used "productive" in this sense in the "Getting Started" section of the main documentation page too. It's less confusing there, since "productive projects" being explicitly used in contrast to "test projects". But (unless I'm misunderstanding) I'd strongly suggest changing it there to "production" - like I said, it's "less confusing there", but I just mean that in the sense of "it's easier to figure out that what you really meant was 'production', not 'productive'".

Anyway, about all those compilation errors in my productive (but not production) code: Am I understanding correctly that these are expected? It looks to me like Testably's MockFileSystem is not a drop-in replacement for TestableIO's MockFileSystem, lacks a bunch of things like "AddDirectory()" and "MockFileData", and so it's not a surprise that existing test code will have to change. That's fine with me, but I have two specific questions based on what I think I've figured out so far:

(1) Is there a way to add stuff after construction? For example, to replace code like:

var fileSystem = new MockFileSystem();
fileSystem.AddDirectory(someDirectoryName);

Obviously in that simple example, it would be easy enough to change it to use the fluent construction syntax that's suggested in "Getting Started", but there are more complex cases where my existing code would be harder to shoehorn into construction time.

(2) Is there a way to deal with non-Windows things (even if the tests are running on a Windows box)? I see the "Getting Started" example relies on WithDrive() on "D:" and InitializeIn() on that drive, which doesn't make sense for Unix-like systems.

Thanks again.

[vbreuss]

Thanks for the feedback about "productive code" vs "production code". I definitely didn't want to cause confusion and will try to find a less confusing wording.
That said, the compilation errors in your test projects are expected. The MockFileSystem is not a drop-in replacement!

(1) Is there a way to add stuff after construction? For example, to replace code like:

var fileSystem = new MockFileSystem();
fileSystem.AddDirectory(someDirectoryName);

Yes, either you directly create it with the "normal" syntax (e.g. fileSystem.Directory.CreateDirectory(someDirectoryName)) or you can use the Initialize extension methods:

var fileSystem = new MockFileSystem();
fileSystem.Initialize().WithSubdirectory(someDirectoryName);

(2) Is there a way to deal with non-Windows things (even if the tests are running on a Windows box)? I see the "Getting Started" example relies on WithDrive() on "D:" and InitializeIn() on that drive, which doesn't make sense for Unix-like systems.

Of course, the whole file system runs on all platforms. The WithDrive extension is completely optional, but you can use it even on Unix-Systems to specify certain hardware-requirements, e.g.

var fileSystem = new MockFileSystem();
fileSystem.WithDrive(d => d.SetTotalSize(5));
fileSystem.File.WriteAllText("foo", "some text longer than 5 bytes"); // this will throw an IOException that there is not enough space on the disk.

If you don't need this functionality, just omit the WithDrive extension.

It would probably be a good idea to improve the README to make this clearer.

[kmsmahu]

@vbreuss I am curious what the reason for the existance of Testably is? Would you mind sharing what the plan for this library is? It seems it is just an enhanced version of TestableIo. Why are the new features not added to TestableIO directly? Thanks in advance.

[vbreuss]

@kmsmahu
Originally I tried to implement the features in TestableIO, but had some blocking issues. Due to the architecture of allowing direct access to the stored entities (MockFileData and MockDirectoryData) it is very hard to keep a consistent state. When I wanted to implement correct update of LastAccessTime and LastUpdateTime it got very complicated. That was the reason, I started with Testably. Here the MockFileSystem is much more restrictive in how you can manipulate it (more or less the normal file system operations), but this allows some advanced scenarios like FileSystemWatcher, correct time updates, working with multiple drives with limited size, simulating the file system on other platforms, etc.

Also the test suite is much more extensive and runs on the build system against the real file system on Linux, Windows and MacOS which means, that the tested scenarios work the same on the mock and the real file system.

The two projects use the same interfaces and for the forseeable future, I will maintain both, but will not invest much time in new functionality in TestableIO.

[kmsmahu]

@vbreuss Thanks for your response. That was quite helpful.

[mgrandi]

I would like to comment that the lack of differences and lack of documentation on what the differences are have thoroughly confused my coworkers, requiring me to download each nuget package to see where stuff comes from. Not to mention that regardless of which one you pick, the namespace for some of the objects is in a third namespace , System.IO

if the API is going to remain the same between them, please put a giant banner readme on the page explaining the differences , or the overall plan for both libraries, because as it stands people are going to pick one or the other, and not realize that TestableIO is less maintained as you mentioned

[vbreuss]

@mgrandi : I fear there is a misunderstanding!

When you compare these project you have to distinguish the API surface in your production projects and in your test projects. By using the same interface dependency, you can switch between Testably.Abstractions.Testing and TestableIO.System.IO.Abstractions.TestingHelpers without adapting your production code. That is also the reason, why I did not change the used namespace (System.IO.Abstractions was used a long time ago).
But the two testing projects have a very different API surface and are not interchangeable.

I will maintain both packages for the forseeable future, but new features will be implemented here (Testably). If the feature set in the TestableIO package is sufficient for you, you can still use this...

[mgrandi]

this is exactly the thing that would be helpful on the readme of both projects

[vbreuss]

I will have to think about how best to formulate this statement, so that it does not cause confusion...

[montoner0]

I think it would be nice to mention in the readme that you can't use Testably just as an improved MockFileSystem for code that uses TestableIO. Because that was my first impression.

[vbreuss]

I am not sure, I understand what you mean, @montoner0:
You can use the MockFileSystem from Testably to test code using TestableIO, but you should not mix both in one test project. Did you mean something different?

Regarding the readme: I am open to any improvements. Feel free to open a PR!