Add ByteRepr for Ptr

[lucic71]

ByteRepr::to_bytes and ByteRepr::from_bytes are implemented using the new PtrRegistry. PtrRegistry is a global collection that keeps mappings between Ptr and [base, base + byte_len]. The right side of the interval includes base + byte_len because it's valid to have a pointer +1 OOB.

When switching from Ptr to the integer representation, the program is free to do arithmetic on the integer representation. When switching back form integer to Ptr, we construct a Ptr if the integer representation is describing a valid Ptr inside PtrRegistry, otherwise panic.

The PtrRegistry interface is defined as follows:

struct PtrRegistry {
    // Builds a synthetic address space that behaves like C's. Neither Weak::as_ptr (1) nor &T (2)
    // can be range bases. A bump cursor inside RangeAllocator hands disjoint, never-reused,
    // [base, base + byte_len] ranges.
    //
    // (1) Weak<RefCell<T>>::as_ptr() returns the address of the Rc payload. That's around 16
    // bytes. This breaks for ranges > 16 bytes as it would overlap with other valid ranges.
    // (2) &T does not live across reallocations, especially when T is Vec<T>
    ranges: RangeAllocator,

    // Mapping between Ptr and [base, base + byte_len] ranges. For efficiency, the mapping is stored
    // as base -> (Ptr, byte_len). Byte::from_bytes is doing O(log N) lookups of SyntheticAddr
    // inside the [base, base + byte_len] ranges.
    entries: BTreeMap<SyntheticAddr, (AnyPtr, ByteLen)>,
}

impl PtrRegistry {
  // Using the real address (Weak::as_ptr) and the length of the pointed-to data, create
  // a synthetic address (stable address) that is used as a key in PtrRegistry::entries
  fn put(&mut self, real_addr: RealAddr, byte_len: ByteLen, ptr: AnyPtr) -> SyntheticAddr;

  // Using a SyntheticAddr, return the associated entry in PtrRegistry::entries. The input
  // addr can be in the middle of [base, base + offset), get returns base. 
  fn get(&self, addr: SyntheticAddr) -> Option<(SyntheticAddr, AnyPtr, ByteLen)>;
}

ByteRepr::to_bytes becomes a call to:

PtrRegistry::put(
    // address() (Weak::as_ptr()) is only an identity key. It cannot act as a range base. This
    // is the real address described above in the interface of PtrRegistry::put. It points
    // at the Rc payload which does not know how many c_byte_len bytes are layed out in
    // memory. For example, the Rc payload is around 16 bytes, but we want to serialize a
    // pointer to object that is 100 bytes long. Doing [address(), address() + 100] is wrong because
    // that can overlap with other valid Rc allocations.
    //
    // PtrRegistry::put takes care of this by creating a SyntheticAddr out of (address(), c_byte_len())
    // that can be used as a non-overlapping range with the correct size.                                                                                    
    self.kind.address(),                                                                                               
    self.c_byte_len(),
    // This is a weak pointer and a type erased pointer at the same time. It will be saved in
    // PtrRegistry::entries. It's rebased at offset 0 becaues when it will be deserialized the
    // correct offset will be applied.                                                                                                   
    Ptr {                                                                                                              
        offset: 0,                                                                                                     
        kind: self.kind.clone(),                                                                                       
    }                                                                                                                  
    .to_any(),                                                                                                         
) 

ByteRepr::from_bytes becomes a call to PtrRegistry::get + reconstructing the Ptr with the correct offset and the correct type.

Notes about this implementation:

1. PtrRegistry::put does O(1) amortized eviction using AnyPtr::is_dangling. Instead of traversing all entries an every put, which would be O(n), we use the following strategy: evict only after entries has doubled in size since the last eviction. If n is the size after the current eviction, we put n elements until we reach 2n, which is the next eviction step. This means that the amortized cost is O(2n/n) = O(1)
2. This model does not panic on pointers crafted from out-of-bounds integers, for example:

int a = 1;
int b[100];
auto p = (uintptr_t) &a;
*(int*)(p + 50) = 42;

&a is serialized as a Ptr with [base_a, base_a + 4], then on the integer representation of the Ptr, the program does base_a + 50 which happens to fit into [base_b, base_b + 100]. Ptr::from_bytes happily deserializes base_a + 50 as being part of the b range. This is not ideal, a panic should be generated instead. @nunoplopes should we focus on fixing this now or can we consider it a known limitation?

3. RangeAllocator always allocates disjoint ranges. So endA == startB is not supported because the right end of the interval [base, base + size] is inclusive in order to accept creating a +1 OOB pointer
4. The integer representation of NULL is always 0
5. Because AnyPtr is saved in PtrRegistry, putting a pointer of type A and getting a pointer of type B is allowed using reinterpret_cast

[nunoplopes]

The code seems to be a bit inefficient. I think a single tree is sufficient. Cleanup should happen during traversal and be limited to the visited nodes. We shouldn't visit the whole tree.

[nunoplopes]

this cursor seems like an unnecessary hack

[nunoplopes]

Also, this implementation doesn't work for the +1 OOB case. The only way to do this properly is with a new Ptr kind.

[lucic71]

Cleanup should happen during traversal and be limited to the visited nodes. We shouldn't visit the whole tree.

I don't see how this evicts all dangling references. Only evicting during traversal means that some dangling references will leave in the tree forever, especially in the current implementation where RangeAllocator does not reuse addresses. Besides this the cost of whole-tree-eviction is O(1) amortized.

this cursor seems like an unnecessary hack

The cursor is here because neither Weak::as_ptr nor &T from Ptr<T> can work as stable ranges.

1. Weak::as_ptr: Imagine having a struct with with char arr[1000]. In C, the struct would have size 1000, in Rust it would have size 8 bytes (the size of Rc in Rc<RefCell<Box<[u8]>>>). Allocating the [base, base + 1000] range for a 8 bytes allocation is wrong because it would overlap with other valid ranges.
2. &T: Imagine having struct big arr[10] where sizeof(struct big) == 1000 with 2 500 bytes long fields. Taking the address of the array inside the Box is still wrong because in C the size of one element is 1000 while in Rust is 16 (2 Rc's). Allocating the [base, base + 1000 * 10] range for a 16 * 10 bytes allocation is wrong because it would overlap with other valid ranges.

So the cursor is there to allow ranges to work over C sizes, not over Rust sizes.

Also, this implementation doesn't work for the +1 OOB case. The only way to do this properly is with a new Ptr kind.

Can you eleborate on that? I don't see how a new Ptr kind can solve this because Ptr does not store any information about layout, only about the data it points to. The current refcount model does not preserve struct layout because each field is a Value. If we want to preserve C layout, I think a possible solution would be something similar to unions where we have a backing byte storage that preserves C layout + reinterpreted view for each member.

[lucic71]

2. This model does not panic on pointers crafted from out-of-bounds integers, for example:

int a = 1;
int b[100];
auto p = (uintptr_t) &a;
*(int*)(p + 50) = 42;

&a is serialized as a Ptr with [base_a, base_a + 4], then on the integer representation of the Ptr, the program does base_a + 50 which happens to fit into [base_b, base_b + 100]. Ptr::from_bytes happily deserializes base_a + 50 as being part of the b range. This is not ideal, a panic should be generated instead. @nunoplopes should we focus on fixing this now or can we consider it a known limitation?

I was thinking a bit more about this and I think I have a (not 100% correct, but practical) solution. RangeAllocator now hands addresses that have a gap of 1 byte between them. Because of this is easy to overflow an object and jump into a new one.

Instead of having a 1 byte gap. We should have a 4GB gap, effectively using a SyntheticAddr with the upper 32 bytes as address (beginning of range), and lower 32 bytes as offset in that range. This way it will be harder for objects to overflow because they will jump into the gap which is direct panic.

The limitation of this is that it limits object size to at most 4GB and if a C program wants to do object + 4.1GB this solutoin fails.

Ideally we would have 64 bytes for address and 64 bytes for offset. But that's harder to achieve because the size of the serialized pointer must stay the same as in C, which is 64 bytes.