Linker reads incorrect string constant as a type name, sometimes erroring if this constant has been protected by Dotfuscator

[Joe-Sewell-PreEmptive]

Steps to Reproduce

Minimal repro:

1. Create a method that accepts 1 string parameter and returns a string representing the name of a type in the project. E.g. string ReturnsTypeName(string x), where the returned value is something like "MyNamespace.MyType".
   • The method may or may not take additional parameters, e.g. string ReturnsTypeName(string x, int y).
2. Call the method and pass the resulting value to Type.GetType. For the argument to the method, use a string literal that has an unpaired UTF-16 surrogate. E.g., var x = Type.GetType(ReturnsTypeName("\uDDD0")).
3. Attempt to build in Release, with the Linker enabled (tested on "Sdk Assemblies Only").

Attached is a ZIP file with this minimal repro.

InvalidUTF16.zip

Actual scenario that triggered this bug filing:

I work for the Dotfuscator team at PreEmptive, a Xamarin partner, and am filing this issue on behalf of my employer.

Dotfuscator supports protecting Xamarin Android projects. However, it seems like a recent update to the Xamarin Android linker has caused it to fail when processing certain kinds of protected code. This issue was originally reported to us by a Stack Overflow question.

The core issue appears to be that the Xamarin linker is analyzing a call in the form Type.GetType(ReturnsTypeName("some_string")) by assuming that "some_string" is the type name being passed to Type.GetType, when in reality it is being passed to another method (ReturnsTypeName) first, which then returns the type name to look up.

Normally this might not be a problem, but Dotfuscator takes advantage of a technical fact of how the .NET runtime treats user strings (those stored in the assembly's #US blob heap). Specifically, while strings are stored as sequences of UTF-16 code units, the runtime allows any sequence of 16-bit values, not just a sequence that is a well-formed UTF-16 string. For instance, .NET (and the C# compiler) allow strings to have unpaired UTF-16 surrogates. Dotfuscator's knowledge of this fact results in scenarios in protected code similar to the minimal repro: an ill-formed UTF-16 string constant is passed to one method, which returns a well-formed UTF-16 string (only known at runtime) for use in reflection.

Based on the stack trace, it looks like the linker is attempting to translate the string constant into UTF-8, and erroring because the constant is ill-formed UTF-16. Thus, this issue might be fixed by handling this error gracefully and/or by not treating "some_string" as a type name in the first place.

This blocks our customers from using our Runtime Checks (e.g., root detection, tamper detection, etc.) features when also using the Xamarin linker. A workaround is to disable the Xamarin linker when using Dotfuscator, but we think some customers will want to have both enabled. Our team is also investigating other possible workarounds for the Runtime Checks issue specifically, but we suspect some customers code may still trigger this scenario with their own code after it has been put through Dotfuscator.

Expected Behavior

The build succeeds, with the linker ignoring the ill-formed UTF-16 string constant (possibly emitting a warning) and continuing to analyze other call sites.

Actual Behavior

The build errors during the LinkAssemblies task, with a message similar to:

2>MSBUILD : error MSB4166: Child node "2" exited prematurely. Shutting down. Diagnostic information may be found in files in "C:\Users\jsewell\AppData\Local\Temp\" and will be named MSBuild_*.failure.txt. This location can be changed by setting the MSBUILDDEBUGPATH environment variable to a different directory.
2>MSBUILD : error MSB4166: C:\Users\jsewell\AppData\Local\Temp\MSBuild_pid-17500_3c3a53c096dc4f9fa9f0261a2d0d8842.failure.txt:
2>MSBUILD : error MSB4166: UNHANDLED EXCEPTIONS FROM PROCESS 17500:
2>MSBUILD : error MSB4166: =====================
2>MSBUILD : error MSB4166: 2020-07-09 5:00:37 PM
2>MSBUILD : error MSB4166: System.Text.EncoderFallbackException: Unable to translate Unicode character \uDDD0 at index 173 to specified code page.
2>MSBUILD : error MSB4166:    at System.Text.EncoderExceptionFallbackBuffer.Fallback(Char charUnknown, Int32 index)
2>MSBUILD : error MSB4166:    at System.Text.EncoderFallbackBuffer.InternalFallback(Char ch, Char*& chars)
2>MSBUILD : error MSB4166:    at System.Text.UTF8Encoding.GetByteCount(Char* chars, Int32 count, EncoderNLS baseEncoder)
2>MSBUILD : error MSB4166:    at System.Text.UTF8Encoding.GetByteCount(String chars)
2>MSBUILD : error MSB4166:    at System.IO.BinaryWriter.Write(String value)
2>MSBUILD : error MSB4166:    at Microsoft.Build.Framework.BuildEventArgs.WriteToStream(BinaryWriter writer)
2>MSBUILD : error MSB4166:    at Microsoft.Build.Framework.LazyFormattedBuildEventArgs.WriteToStream(BinaryWriter writer)
2>MSBUILD : error MSB4166:    at Microsoft.Build.Framework.BuildMessageEventArgs.WriteToStream(BinaryWriter writer)
2>MSBUILD : error MSB4166:    at Microsoft.Build.Shared.LogMessagePacketBase.WriteToStream(ITranslator translator)
2>MSBUILD : error MSB4166:    at Microsoft.Build.Shared.LogMessagePacketBase.Translate(ITranslator translator)
2>MSBUILD : error MSB4166:    at Microsoft.Build.BackEnd.NodeEndpointOutOfProcBase.RunReadLoop(Stream localReadPipe, Stream localWritePipe, ConcurrentQueue`1 localPacketQueue, AutoResetEvent localPacketAvailable, AutoResetEvent localTerminatePacketPump)

Version Information

Microsoft Visual Studio Professional 2019
Version 16.6.3
VisualStudio.16.Release/16.6.3+30225.117
Microsoft .NET Framework
Version 4.8.03752

Installed Version: Professional

Visual C++ 2019   00435-60000-00000-AA989
Microsoft Visual C++ 2019

ASP.NET and Web Tools 2019   16.6.954.58314
ASP.NET and Web Tools 2019

ASP.NET Core Razor Language Services   16.1.0.2020603+b3ac44798c16fff5b95dbcfe62dea84aa9a1bd72
Provides languages services for ASP.NET Core Razor.

ASP.NET Web Frameworks and Tools 2019   16.6.954.58314
For additional information, visit https://www.asp.net/

Azure App Service Tools v3.0.0   16.6.954.58314
Azure App Service Tools v3.0.0

Azure Functions and Web Jobs Tools   16.6.954.58314
Azure Functions and Web Jobs Tools

C# Tools   3.6.0-4.20251.5+910223b64f108fcf039012e0849befb46ace6e66
C# components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.

Common Azure Tools   1.10
Provides common services for use by Azure Mobile Services and Microsoft Azure Tools.

Dotfuscator Community Edition   5.44.0.10087-6530a8d80a
PreEmptive Protection - Dotfuscator CE

Extensibility Message Bus   1.2.0 (d16-2@8b56e20)
Provides common messaging-based MEF services for loosely coupled Visual Studio extension components communication and integration.

IntelliCode Extension   1.0
IntelliCode Visual Studio Extension Detailed Info

JetBrains ReSharper Ultimate 2019.3.4   Build 193.0.20200226.112949
JetBrains ReSharper Ultimate package for Microsoft Visual Studio. For more information about ReSharper Ultimate, visit http://www.jetbrains.com/resharper. Copyright © 2020 JetBrains, Inc.

Microsoft Azure Tools   2.9
Microsoft Azure Tools for Microsoft Visual Studio 2019 - v2.9.30212.1

Microsoft Continuous Delivery Tools for Visual Studio   0.4
Simplifying the configuration of Azure DevOps pipelines from within the Visual Studio IDE.

Microsoft JVM Debugger   1.0
Provides support for connecting the Visual Studio debugger to JDWP compatible Java Virtual Machines

Microsoft Library Manager   2.1.50+g25aae5a24a.R
Install client-side libraries easily to any web project

Microsoft MI-Based Debugger   1.0
Provides support for connecting Visual Studio to MI compatible debuggers

Microsoft Visual C++ Wizards   1.0
Microsoft Visual C++ Wizards

Microsoft Visual Studio Tools for Containers   1.1
Develop, run, validate your ASP.NET Core applications in the target environment. F5 your application directly into a container with debugging, or CTRL + F5 to edit & refresh your app without having to rebuild the container.

Microsoft Visual Studio VC Package   1.0
Microsoft Visual Studio VC Package

Mono Debugging for Visual Studio   16.6.17 (9692114)
Support for debugging Mono processes with Visual Studio.

NuGet Package Manager   5.6.0
NuGet Package Manager in Visual Studio. For more information about NuGet, visit https://docs.nuget.org/

ProjectServicesPackage Extension   1.0
ProjectServicesPackage Visual Studio Extension Detailed Info

ResXManager   1.42.3648.0
Manage localization of all ResX-Based resources in one place. Shows all resources of a solution and let's you edit the strings and their localizations in a well-arranged data grid.

SQL Server Data Tools   16.0.62006.03190
Microsoft SQL Server Data Tools

Test Adapter for Boost.Test   1.0
Enables Visual Studio's testing tools with unit tests written for Boost.Test.  The use terms and Third Party Notices are available in the extension installation directory.

Test Adapter for Google Test   1.0
Enables Visual Studio's testing tools with unit tests written for Google Test.  The use terms and Third Party Notices are available in the extension installation directory.

TypeScript Tools   16.0.20417.2002
TypeScript Tools for Microsoft Visual Studio

Visual Basic Tools   3.6.0-4.20251.5+910223b64f108fcf039012e0849befb46ace6e66
Visual Basic components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.

Visual F# Tools 10.9.1.0 for F# 4.7   16.6.0-beta.20217.4+1c969cac25e2d38d71872efe6c8226029e42bb59
Microsoft Visual F# Tools 10.9.1.0 for F# 4.7

Visual Studio Code Debug Adapter Host Package   1.0
Interop layer for hosting Visual Studio Code debug adapters in Visual Studio

Visual Studio Container Tools Extensions (Preview)   1.0
View, manage, and diagnose containers within Visual Studio.

Visual Studio Tools for CMake   1.0
Visual Studio Tools for CMake

Visual Studio Tools for Containers   1.0
Visual Studio Tools for Containers

VisualStudio.DeviceLog   1.0
Information about my package

VisualStudio.Mac   1.0
Mac Extension for Visual Studio

Xamarin   16.6.000.1062 (d16-6@41e1f0e)
Visual Studio extension to enable development for Xamarin.iOS and Xamarin.Android.

Xamarin Designer   16.6.0.318 (remotes/origin/d16-6@66afc5421)
Visual Studio extension to enable Xamarin Designer tools in Visual Studio.

Xamarin Templates   16.6.40 (1f1466f)
Templates for building iOS, Android, and Windows apps with Xamarin and Xamarin.Forms.

Xamarin.Android SDK   10.3.1.4 (d16-6/3a10de9)
Xamarin.Android Reference Assemblies and MSBuild support.
    Mono: 165f4b0
    Java.Interop: xamarin/java.interop/d16-6@2cab35c
    ProGuard: xamarin/proguard/master@905836d
    SQLite: xamarin/sqlite/3.31.1@49232bc
    Xamarin.Android Tools: xamarin/xamarin-android-tools/d16-6@bfb66f3


Xamarin.iOS and Xamarin.Mac SDK   13.18.2.1 (29c4ea7)
Xamarin.iOS and Xamarin.Mac Reference Assemblies and MSBuild support.

Log File

(Included is an MSBuild log, not an ADB log.)

VS-detailed-log.txt
MSBuild_pid-17500_3c3a53c096dc4f9fa9f0261a2d0d8842.failure.txt

[Joe-Sewell-PreEmptive]

Also, I think this might better be categorized as "Area: Linker", but I don't think I can change that myself.

[mina5500]

same.... please fix this asap

[radekdoulik]

I am unable to reproduce with VS 16.7, so hopefully it has been fixed meanwhile. Could you please confirm that it works for you with 16.7 as well?

I think the problem is/was not in the linker itself, but rather msbuild tool, which crashed when logging the output messages from the linker about the reflection calls.

Locally the build completes and I see:

...
  Reflection call 'System.Type System.Type::GetType(System.String)' inside 'System.Type Java.Interop.__TypeRegistrations::Lookup(System.String[],System.String)' was detected with argument which cannot be analyzed (TaskId:243)
  Reflection call 'System.Reflection.FieldInfo System.Type::GetField(System.String,System.Reflection.BindingFlags)' inside 'Xamarin.Forms.BindableProperty Xamarin.Forms.VisualElement::Xamarin.Forms.StyleSheets.IStylable.GetProperty(System.String,System.Boolean)' was detected with argument which cannot be analyzed (TaskId:243)
  Reflection call 'System.Type System.Type::GetType(System.String)' inside 'System.Void InvalidUTF16.MainPage/<Button_OnClicked>d__1::MoveNext()' was detected with type name `E` which can't be resolved. (TaskId:243)
  Reflection call 'System.Type System.Type::GetType(System.String)' inside 'System.Void InvalidUTF16.MainPage/<Button_OnClicked>d__1::MoveNext()' was detected with type name `?` which can't be resolved. (TaskId:243)
  Reflection call 'System.Type System.Type::GetType(System.String,System.Boolean)' inside 'System.Object Android.Runtime.AndroidEnvironment::GetHttpMessageHandler()' was detected with argument which cannot be analyzed (TaskId:243)
...

[mina5500]

It is not working with Latest VS 16.7.2

Dotfuscator is enabled in release mode
Xamarin Forms Android Project

Once the Linker is set to none the issue is gone but we can't go with this workaround as the application size is doubled.

[radekdoulik]

Interesting, I can now replicate it as well. It happens in UI only and does not on the command line.

@joj could you please take a look? is it UI specific or msbuild problem?

[joj]

@radekdoulik I'm seeing the same error in command-line. The error is not shown (unless you're on a diagnostic build and even then it's difficult to find), but if you go to temp you'll see a new MsBuild_ file with the same error. I have a couple of binlogs (one from VS and one from command line) but they don't look very useful. The only difference between VS and command-line when building is that VS builds each project, whereas command-line builds the solution. This results in slightly different build orders sometimes, but I'm not seeing that here.

[radekdoulik]

@joj for me it passes the problematic output line (#4909 (comment)) and the build succeeds.

my msbuild version in the terminal:

MSBuild.exe /version
Microsoft (R) Build Engine version 16.7.0+b89cb5fde for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

16.7.0.37604

how does the IDE handles the build? could I have different msbuild version in the terminal compared to what my IDE uses?

the stacktrace looks like a crash in msbuild, I am not sure though whether it might be originating in UI.

[joj]

As long as you're using the right command line prompt, you should have the same msbuild. Make sure you're rebuilding, and in release (/p:Configuration=Release /t:Rebuild). The build will generate artifacts, so a simple build will not repro the error. I'm on 16.7.0.27905 right now, but I really don't think msbuild is the one failing here.

[joj]

Also, I'm seeing the build succeed... but then I have the new error file and if I generate a binlog for it I can see the same issue VS has.

[joj]

OTOH, if msbuild is crashing on a log, then it may make sense to add /v:diag so you can have the full log and see the crash? MSBuild might have changed something or we may be logging incorrectly at some point, too.