ByteBuffer is not exposing the Array() method

[ajpinedam]

Steps to Reproduce

1. Create a ByteArray object allocating any number of items
   var byteBuffer = ByteBuffer.Allocate(2).PutShort(timeout);
2. byteBuffer should have an Array() method (or Property) to get the array from the ByteBuffer object but it's not.

byteBuffer.Array()

Java documentation: https://docs.oracle.com/javase/7/docs/api/java/nio/ByteBuffer.html

C# / Xamarin.Android Doc: https://docs.microsoft.com/en-us/dotnet/api/java.nio.bytebuffer?view=xamarin-android-sdk-9

Expected Behavior

Array method is present in the ByteBuffer object

Actual Behavior

Array method is not present in the ByteBuffer object

Version Information

Visual Studio Community 2019 for Mac
Version 8.7.9 (build 9)
Installation UUID: 68a8f36e-286f-4402-ad89-4d3d8e5c8fe8
GTK+ 2.24.23 (Raleigh theme)
Xamarin.Mac 6.18.0.23 (d16-6 / 088c73638)

Package version: 612000093

Details

Mono Framework MDK Runtime: Mono 6.12.0.93 (2020-02/620cf538206) (64-bit) Package version: 612000093

Roslyn (Language Service)
3.7.0-6.20427.1+18ede13943b0bfae1b44ef078b2f3923159bcd32

NuGet
Version: 5.7.0.6702

.NET Core SDK
SDK: /usr/local/share/dotnet/sdk/3.1.403/Sdks
SDK Versions:
3.1.403
3.1.402
3.1.300
3.1.202
3.1.200
3.1.101
3.1.100
3.0.101
3.0.100
2.2.402
2.1.701
2.1.700
2.1.505
2.1.504
2.1.503
2.1.302
MSBuild SDKs: /Library/Frameworks/Mono.framework/Versions/6.12.0/lib/mono/msbuild/Current/bin/Sdks

.NET Core Runtime
Runtime: /usr/local/share/dotnet/dotnet
Runtime Versions:
3.1.9
3.1.8
3.1.4
3.1.2
3.1.1
3.1.0
3.0.1
3.0.0
2.2.7
2.1.23
2.1.22
2.1.18
2.1.16
2.1.15
2.1.14
2.1.13
2.1.12
2.1.11
2.1.9
2.1.8
2.1.7
2.1.2

Xamarin.Profiler
Version: 1.6.12.29
Location: /Applications/Xamarin Profiler.app/Contents/MacOS/Xamarin Profiler

Updater
Version: 11

Apple Developer Tools
Xcode 12.1 (17222)
Build 12B45b

Xamarin.Mac
Version: 6.20.2.2 (Visual Studio Community)
Hash: 817b6f72a
Branch: d16-7
Build date: 2020-07-18 18:44:59-0400

Xamarin.Android
Version: 11.0.2.0 (Visual Studio Community)
Commit: xamarin-android/d16-7/025fde9
Android SDK: /Users/andrespineda/Library/Developer/Xamarin/android-sdk-macosx
Supported Android versions:
4.4 (API level 19)
5.1 (API level 22)
7.1 (API level 25)
8.0 (API level 26)
8.1 (API level 27)

SDK Tools Version: 26.1.1
SDK Platform Tools Version: 30.0.0
SDK Build Tools Version: 29.0.2

Build Information:
Mono: 83105ba
Java.Interop: xamarin/java.interop/d16-7@1f3388a
ProGuard: Guardsquare/proguard@ebe9000
SQLite: xamarin/sqlite@1a3276b
Xamarin.Android Tools: xamarin/xamarin-android-tools/d16-7@017078f

Microsoft OpenJDK for Mobile
Java SDK: /Users/andrespineda/Library/Developer/Xamarin/jdk/microsoft_dist_openjdk_1.8.0.25
1.8.0-25
Android Designer EPL code available here:
https://github.com/xamarin/AndroidDesigner.EPL

Android SDK Manager
Version: 16.7.0.13
Hash: 8380518
Branch: remotes/origin/d16-7~2
Build date: 2020-09-16 05:12:24 UTC

Android Device Manager
Version: 16.7.0.24
Hash: bb090a3
Branch: remotes/origin/d16-7
Build date: 2020-09-16 05:12:46 UTC

Xamarin Designer
Version: 16.7.0.495
Hash: 03d50a221
Branch: remotes/origin/d16-7-vsmac
Build date: 2020-08-28 13:12:52 UTC

Xamarin.iOS
Version: 14.2.0.12 (Visual Studio Community)
Hash: e0ce49680
Branch: xcode12.1
Build date: 2020-10-20 09:52:06-0400

Build Information
Release ID: 807090009
Git revision: d29d54665a1fc79986687a540a7a4676f97ad3da
Build date: 2020-10-21 15:05:18-04
Build branch: release-8.7
Xamarin extensions: d29d54665a1fc79986687a540a7a4676f97ad3da

Operating System
Mac OS X 10.15.7
Darwin 19.6.0 Darwin Kernel Version 19.6.0
Mon Aug 31 22:12:52 PDT 2020
root:xnu-6153.141.2~1/RELEASE_X86_64 x86_64

Log File

[jpobst]

Looks like we explicitly removed it 10+ years ago: https://github.com/xamarin/xamarin-android/blob/4cd3913def29cb445e07a1e0c3a0e03877ad8ee8/src/Mono.Android/metadata#L473.

I'll need to investigate a bit to see if I can figure out why.

You can probably work around it using one of:

Get(byte[] dst);
Get(byte[] dst, int offset, int length);

[jonpryor]

I'll need to investigate a bit to see if I can figure out why.

I suspect part of the problem is our old favorite covariant return types: ByteByffer.array() overrides Buffer.array(), which is abstract:

abstract class Buffer {
    public abstract Object array();
}
class ByteBuffer extends Buffer {
    public byte[] array() {…}
}

What should a binding for this look like? I can think of only three solutions:

1. Java.Lang.Object all the things!
2. Hide Buffer.array(), bind ByteBuffer.array() and other overrides "normally"
3. [Requires prototyping, testing, sanity checking]: pursue one of our long-standing binding ideas of "unifying" java.lang.Object and System.Object -- emitting object everywhere java.lang.Object exists -- then using C#9 covariant return types in the override.

Java.Lang.Object all the things!

This "works", but really results in a fugly binding:

abstract class Buffer {
    public abstract Java.Lang.Object Array();
}
class ByteBuffer : Buffer {
   public override Java.Lang.Object Array();
}

Hide Buffer.array(), bind ByteBuffer.array() and other overrides "normally"

This makes for a "nicer" API, but means Buffer in-and-of-itself is a potentially "less useful" abstraction:

abstract class Buffer {
    public abstract Java.Lang.Object Array();
}
class ByteBuffer : Buffer {
   public byte[] Array();
}

C#9!

C#9 adds covariant return types!

This is a more "pie-in-the-sky" idea, but What If™

abstract class Buffer {
    public abstract object Array();
}
class ByteBuffer : Buffer {
   public override byte[] Array();
}

This assumes that C#9 covariant return types can have ByteBuffer.Array() return a byte[] when the base method returns object. I don't know if this is actually supportable.

Then there's the "ABI break" concern: we can't bind all instances of java.lang.Object with System.Object, it'll break everything, so if we wanted to consider this we'd have to only do this for members added since API-30.

(Then there's the other question of what happens with older compilers trying to use this…)

[ajpinedam]

What a great explanation @jonpryor ! Thanks.

Just wondering if we can avoid breaking things by creating an Extension method (or any Utility method) the same way we have done in other cases (I don't have any example now but sure that I have seen it).

This way we can leave things as they are in terms of the Java APIs but expose our own implementation that will return the desired value.

btw: I tried @jpobst suggestion (Thanks for this)

Get(byte[] dst);

but I am having an exception of type Java.Nio.BufferUnderflowException

This is what I am doing:

var byteBuffer = ByteBuffer.Allocate(2).PutShort(1);
byte[] result = new byte[10];
byteBuffer.Get(result);

I am getting the same exception even while using just

var tbyte = byteBuffer.Get();

[jpobst]

I think you are trying to read from the end of the buffer. You will need to "rewind" so that it is pointed at the beginning:

var byteBuffer = ByteBuffer.Allocate (10).PutShort (33);
var result = new byte [byteBuffer.Capacity ()];

byteBuffer.Rewind ();

byteBuffer.Get (result);

[ajpinedam]

Thank you @jpobst .

[andreinitescu]

I think you are trying to read from the end of the buffer. You will need to "rewind" so that it is pointed at the beginning:

var byteBuffer = ByteBuffer.Allocate (10).PutShort (33);
var result = new byte [byteBuffer.Capacity ()];

byteBuffer.Rewind ();

byteBuffer.Get (result);

If I understand correctly, without the array() method, in order to access the buffer of a ByteBuffer instance, I have to allocate and copy to an intermediary byte buffer (in your example, it's the result buffer).

Due to performance reasons, I'd like to avoid the allocation and copying, is it possible?
Stupid question, can I make a binding myself for the array() method somehow?

[jpobst]

If you want to access the array in C# code, yes, you will need to allocate memory in C# and copy the array there. If there were an array() method binding it would do exactly this for you, but it would still allocate the memory in C# and copy.

Note that it's actually worse than that. If you then want the ByteBuffer to have the changes you made in C# you will need to copy your changes back to the ByteBuffer.

The problem is that the backing byte[] lives in the Java VM. In order to manipulate it in the .NET runtime you either have to copy it to .NET runtime and then copy it back, or you can manipulate it purely using the Java methods defined on ByteBuffer like the various get* and put* methods.

[andreinitescu]

@jpobst Thanks. But what if the ByteBuffer was directly allocated? (ByteBuffer.directAllocate()) I was hoping that the combination of direct memory + being able to access that memory by the array() would have avoided any allocation and copying.

[jpobst]

No, there is no way for the .NET runtime to directly access memory owned by the Java runtime.

A few other options that may or may not help:

• If you are starting from a finished C# byte[], you can call ByteBuffer.Wrap (byte[]) which will only copy the array once (when it copies it from C# to Java to the initial ByteBuffer). Note that after the initial copy, modifying it on the C# side will have no effect so this only helps if you no longer need to access it from C#.
• You can write a method that does all the buffer manipulation in Java and does not expose the buffer to C#, and then bind and wrap that to call from C#. (ie: MyJavaClass.doBufferManipulation (ByteBuffer buf))