API change ideas for elegant mocking

[Gerschtli]

Hey,
what do you think of adding a completly new API to improve readability of mockings. I would suggest something like this:

when(namespace::function)
    .expected(times(3))
    .called_with(
        is(equal_to(5)),
        is(none())
    )
    .returns(7);

A call to expected would be optional and defaulting to times(1). Other parameter values could be at(2), al_least(3) etc.

called_with would be a method accepting a hamcrest2 matcher for each parameter. Internal we could simply test these with assert_that!(expected_value, <matcher>). This method call would also be optional, defaulting to empty parameter list.

returns will simply specify, what the function returns. Alternatively runs_original() will delegate the call to the original implementation.

The only thing what I need to call this a full mocking library would be the ability to mock the behavior for struct instances, like:

struct Struct(u8);

#[mockable]
impl Struct {
    fn get(&self) -> u8 {
        self.0
    }
}

fn test() {
    let a = Struct(1);
    let b = Struct(2);

    when(Struct::get).on(&a).returns(10);

    assert_that!(a.get(), is(equal_to(10)));
    assert_that!(b.get(), is(equal_to(2)));
}

It would be a nice feature to be able to configure (via parameter, function call, etc) if calls not specified in expected raise an error and let the test fail, like:

#[mockable]
fn function(a: u32) -> u32 {
    a
}

#[test]
fn test() {
    // order of mocks matter: at first call param is 1, at second param is 2
    when(function)
        .called_with(is(equal_to(1)))
        .returns(5);
    when(function)
        .called_with(is(equal_to(2)))
        .returns(7);
        
    // something like this, to specify mock type
    fail_on_unmocked_calls(function);

    assert_that!(function(1), is(equal_to(5)));
    assert_that!(function(2), is(equal_to(7)));
    
    // following call should let the tast fail
    assert_that!(function(3), is(equal_to(3)));
}

For struct instances, you could specify with anything like the following, where the test would fail on any call to a method of that struct instance.

let a = Struct(1);
fail_on_unmocked_calls(&a);

These are just some ideas inspired by Mockito, PHPUnit and similar mocking libraries. What do you think of it? Would it be something you want to add in this crate?

I would love to help and contribute to your project if so.

[Gerschtli]

An idea for the struct instance mocking:

#![mockable]

struct Struct(u8);

impl Struct {
    fn get(&self) -> u8 {
        self.0
    }
}

via macro magic complied to this:

struct StructOriginal(u8);

impl StructOriginal {
    fn get(&self) -> u8 {
        self.0
    }
}

struct Struct {
    id: ::uuid::Uuid,
    original: Struct,
}

impl Struct {
    fn get(&self) -> u8 {
        // your function header injection here where you could test on instance equality via
        // self.id, then remove &self from params list on let `called_with` handle the rest

        self.original.get()
    }
}

The problem would be the direct access to properties (pattern matching, constructing via something like Struct(14) etc.). You would need to delegate that to the underlying object, preferably with a previously defined mocked state. The instantiation without a constructor seems to be very tricky, don't have any idea for that.

Needed to write this down this concise, so I don't forget it :D I'm just really excited on having a full-fledged mocking library for rust!

[Gerschtli]

Oh, I have a better idea than my last comment. The following works with your latest release:

pub(crate) struct Struct(u8);

#[mockable]
impl Struct {
    pub(crate) fn get(&self) -> u8 {
        self.0
    }
}

#[test]
fn test_instance_mock() {
    let a = Struct(1);
    let b = Struct(2);

    println!("address a:       {:?}", &a as *const _);
    println!("address b:       {:?}", &b as *const _);

    unsafe {
        Struct::get.mock_raw(|instance| {
            println!("address in call: {:?}", instance as *const _);
            if instance as *const _ == &a as *const _ {
                MockResult::Return(42)
            } else {
                MockResult::Continue((instance,))
            }
        });
    }

    assert_eq!(a.get(), 42);
    assert_eq!(b.get(), 2);
}

which results in following output:

running 1 test
address a:       0x7f86d50f744e
address b:       0x7f86d50f744f
address in call: 0x7f86d50f744e
address in call: 0x7f86d50f744f
test_instance_mock ... ok

It is actually possible (based on my little research and testing) with your implementation as base to build what I proposed.

[CodeSandwich]

I'm very sorry, but since a few days and for a few more I will not be able to work on Mocktopus. I will gladly come back to coding, reviewing and discussing when my personal situation gets a bit more stable and less time consuming.

[asomers]

I agree with @Gerschtli . Mocktopus needs an expectation API before it can be considered a full-featured mocking library. It should be able to expect that a function is called with certain arguments a certain number of times. It should also be able to expect that different methods, or different invocations of the same method, are called in a certain order.

[CodeSandwich]

This is matter of taste, but personally I'm not a fan of verbose API, you've proposed in the first comment. I've been using similar tools in Java and C# and it was very frustrating, I was plowing through docs for hours to understand or compose even relatively simple mocks.

The proposed API needs some refinement and probably will get much more complex:

1. expected must create a handler for asserting call count. It also must be called manually, because automatic invocation in handler's drop will terminate the whole process on assert failure.
2. called_with needs to define which parameter should be tested
3. called_with receives variable number of arguments, which is not supported in Rust, it should probably be a slice or vector with filters
4. called_with implicitly joins filters with logical "or", did I mention plowing through docs to undestand simple mocks?
5. returns works only with Cloneable items, it will have to be extended to work with other types as well. Probably a collection of arbitrary items would do it, but what will happen when they run out? Should we panic or should we stop mocking? (plooowiiing!)

Unfortunately the first proposition of per-object mocking seems completely unrealistic, I have no idea how could we tackle the technical difficulties. The second one makes a lot of sense, but it requires immovability of the object. If we bake pointer comparison into an API, it probably should require object to be Pinned. It probably would be wise to alternatively allow custom comparison, because some structures can be actually unique in some sense (id field or something).

I may seem negative, but I've intentionally avoided this route of Mocktopus API. I've decided that a trivial API and usage of regular Rust logic is better than huge toolkit of complex, use-case-specific functions. Less maintenance, lower learning curve and unlimited elasticity.

BUT

I'm planning to switch mock_safe and mock_raw from accepting only FnMuts to more generic

trait Mock<T, O> {
    fn call_mock(&mut self, args: T) -> MockResult<T, O>;
}

That means 2 things:

1. You will be able to create your own mock API with blackjack and hookers built on top of unmodified Mocktopus for everyone to use
2. Base Mocktopus can start aggregating implementations of Mock for numerous types to create generic building blocks making mocking more pleasant. I'm thinking about collections of returned values, vectors of FnOnces, custom structs for combinators, filters, call counters, etc. I personally wish it won't end up being an opaque complex mess requiring plowing through docs to understand basic mocks.

[Gerschtli]

I understand what you mean but in contrast to your point of view I think an easily readable test should be a key feature of a mocking/testing framework. Tests are meant to be living documentation, to document the expected behaviour and ensure functionality. The main goal is for me a more readable and understandable test,
Additionally that plowing you described would be a bit compensated with the current support of autocompletion and intelli sense. It should be a key requirement to design the api as intuitive as possible through consistent naming conventions etc.

for point 1: I am currently experimenting with python like method decorators to build fixtures. This could be easily be implemented as such. Needs more testing of course.

for point 2 and 3: it could be handled with a macro. How to build it to be the most intuitive is another discussion.

for point 4: Why not use the matcher of https://github.com/Valloric/hamcrest2-rust ?

for point 5: We could also accept Fn, FnOnce and/or FnMut and allow a clonable object as convenience option. On an unmocked call simply fail the test because what else should happen :D

The per object mocking is apart from the technical difficulties a very interesting feature. Even if we include the custom comparison to test for the correct instance.

Of course if you don't want to build such a framework, it would be totally fine for me to create a library around mocktopus.