Contribution idea: Port Orleans to coreclr

[gabikliot]

Port the Orleans code to run on coreclr.
Some APIs from the full .NET got deprecated in coreclr (aka .NET Core), mainly around files and reflection, but at large the porting effort shouldn't be too big. This will allow to run Orleans efficiently across different platforms.

---

Tools:
Coreclr api search
.NET API Usage
.NET Portability Analyzer - download and Install
.NET Portability Analyzer Usage

Progress so far:
CoreCLR milestones
ApiPortabilityAnalysis result file for all projects (let's find an appropriate place to place updates of this file, please don't pollute the master branch with these big reports)

---

There are 2 types of issues:
(a) easy, just change an API - we have done all those.
(b) deeper profound issues where API was deprecated or changed significantly and we will need to investigate other ways to achieve the same functionality:

Need to investigate and consider profound rewrite - feel to open a GH issue with ideas on how to handle each one of those issues (separate issue for each) :
High priority

• Create a v2 branch that changes target for Orleans to .NET Standard 1.6
  • Ideally still use csproj as in https://docs.microsoft.com/en-us/dotnet/articles/core/tutorials/target-dotnetcore-with-msbuild (this means that we'll have to update from .NET 4.5.1 in all dependent projects)
• Remove usage of App Domains for scanning https://github.com/dotnet/coreclr/issues/919
  • Revisit Assembly Loading (LoadFrom -> AsemblyLoadContext, Location,...) Assembly Catalog #2025
• Provide an alternative fallback serializer (BinaryFormatter is not supported in .NET Core). The biggest concern is serializing exceptions, as they are not typically part of the grain interfaces, and we don't generate Orleans serializers for all exceptions in the system. We have a few workarounds in prototype that need to be enhanced.
• PerformanceCounter - not supported in .NET Core.
• Environment.UserName
• CallContext.LogicalSetData - potentially replace with AsyncLocal<T>
• Update codegen to be more aggressive and generate serializers for Exception types
• SerializationInfo - not supported in .NET Core.

2nd priority

• Compile time codegen working with new dotnet cli tooling (although be aware that the tooling will drop support for project.json once it RTMs, and we are not sure if the build tools will work once they start using csproj once again)
  • Avoid having OrleansCodeGenerator depend on Orleans project, since that creates a circular dependency to getting codegen to work easily (especially since Orleans.dll requires build time codegen). This is a nice to have, but not required if we solve the previous item in some other way.
• Allow TestCluster to work without creating new AppDomains for each silo (for example, by leveraging containers, or spawning and managing new processes)

3rd priority

• Separate test projects for each provider that brings in a new dependency (ideally even if that dependency is also supported in .NET Core, otherwise we would need to port the entire codebase before we can start testing each individual component we port)
• CI servers run tests in many platforms
• System.Runtime.Serialization.FormatterServices.GetUninitializedObject
• System.Diagnostics.Process.Handle -> System.Diagnostics.Process.SafeHandle

---

Already done issues (for progress tracking):

• Type.IsInterface-> Type.GetTypeInfo.IsInterface (IsClass, IsGenericType , IsAbstract, IsPrimitive, IsEnum).
• Type -> TypeInfo for most members (in Orleans code, and code generated by OrleansCodeGenerator) CoreCLR Support: fix #1782 - use TypeInfo instead of Type in codegen #1783 Fix several incorrect usages of Type and TypeInfo #1894 Avoid using Type.IsEquivalentTo on code changed by #1963 #1997
• TextReader.Close -> TextReader.Dispose Coreclr compatibility: changed close to dispose & changed of scoping with using #609 Change TextWriter.Close to Dispose #2034
• Socket.Close -> Socket.Dispose
• Delegate.CreateDelegate -> MethodInfo.CreateDelegate
• ApplicationException(string)
• System.Data and System.Transactions - used in SQL utils. Create a separate OrleansSQLUtils project and move all Relational Storage code there.
• XmlTextWriter and XMLNode.SelectNodes.
• Dns.GetHostAddresses.
• System.String.ToLower(CultureInfo.InvariantCulture) -> Use CultureInfo.TextInfo.ToLower(String) instead
• System.UriTypeConverter.
• System.CodeDom.CodeTypeParameter.
• BinaryFormatter.Serialize - solved via Coreclr compatibility: Implemented Json.Net fallback serializer #1047.
• System.Management.ManagementObjectCollection - not supported in .NET Core. Investigate and propose alternative options (if ManagementObject itself is supported). We use it now in one place, to find TotalPhysicalMemory. Solved via Coreclr compatibility: Ifdeffing non-coreclr RAM perf counter #1048

[stavro]

We run the compatibility tool some time ago. There are a couple of unsupported APIs, mainly about files (need to use streams instead) and code gen. We don't think it would be too hard to do the port. Perhaps a good way to start is to re-evaluate what needs to be ported exactly and then do the port in small steps, one API at a time (as opposite to one huge PR). That would make it much easier to validate and accept it, and also other will be able to join in and help.

Can someone share the instructions of how to set up the compatibility tool locally?

[gabikliot]

http://blogs.msdn.com/b/dotnet/archive/2014/08/06/leveraging-existing-code-across-net-platforms.aspx
https://visualstudiogallery.msdn.microsoft.com/1177943e-cfb7-4822-a8a6-e56c7905292b

[pherbel]

Hi
I think it is very important step and I would like to use Orleans client from the coreclr project.
Is there a working in progress branch for this?
Maybe I can contribute

[gabikliot]

We do not have a progress branch for this yet.
In the meanwhile, we are doing a bunch of preparation work that will make the porting easier, such as reducing code generation (which is one of the biggest hurdles for coreclr port). This is tracked here: #474.

If you can contribute to the coreclr port, that would be wonderful! Perhaps a good first step is to precisely identify all pieces that need to be ported.

[sergeybykov]

No branch yet. We can create one for this.

[pherbel]

As I checked the code with the portability analyzer first step could be change of the usage of the Type class.
The analyzer suggests use TypeInfo instead of Type which is fine for the coreclr.
I assume that We don't need to change the CodeGeneration part because it will be deprecated and it has lot of Type usage. Will the CodeGeneration part be also deprecated in the Orleans.dll?

What do you think?