Tips & Tricks: Object-oriented Sorting in JS++ with IComparable<T>

JS++ makes object-oriented sorting easy with the IComparable<T> interface and the Comparison enumeration for type-safe (and readable) comparisons.

Here’s the code. (Don’t worry; I’ll dissect it.)

import System;

class Employee : IComparable<Employee>
{
    private string firstName;
    private string lastName;

    public Employee(string firstName, string lastName) {
        this.firstName = firstName;
        this.lastName = lastName;
    }

    public Comparison compare(Employee that) {
        // Sort by employee surname
        return this.lastName.compare(that.lastName);
    }

    public override string toString() {
    	return this.firstName + " " + this.lastName;
    }
}

Employee zig  = new Employee("Zig", "Ziglar");
Employee john = new Employee("John", "Smith");
Employee abe  = new Employee("Abe", "Lincoln");

Employee[] employees = [ zig, john, abe ];
employees.sort();
Console.log(employees.join(", "));

// Output:
// Abe Lincoln, John Smith, Zig Ziglar

This is beautiful, object-oriented code. All of the custom sorting logic is one line of code. Let’s break down how that happens step-by-step.

1. Implement IComparable<T>

The first step is to implement the IComparable<T> interface. The interface provides only one method to implement: compare.

compare expects the Comparison enumeration as a result. As we can see from the documentation, Comparison can have three possible results: LESS_THAN, GREATER_THAN, and EQUAL. While Java/C# expect -1, 0, and 1, JS++ gives you type-safe and readable comparisons.

IComparable<T> and Comparison form the basis for custom sorting.

2. Determine how to sort

We want to sort Employee objects based on the employee’s last name. In order to do this, we want to compare strings and sort in alphabetical order. While we can do this manually, the JS++ Standard Library already provides these comparisons for us for primitive types.

All primitive types in JS++ are auto-boxed. (Don’t worry, it gets optimized away.) In addition, all primitive types implement IComparable<T> (which provides the compare method).

Thus, since all primitive types provide the compare method, sorting is as easy as this one line of code:

return this.lastName.compare(that.lastName);

This is calling the System.String.compare method, which compares strings lexicographically (in alphabetical order). (Likewise, if you wanted to compare by employee ID number, you might declare an unsigned int and use System.UInteger32.compare.)

Thus, our sorting code and implementation of IComparable<T>.compare is just:

public Comparison compare(Employee that) {
    // Sort by employee surname
    return this.lastName.compare(that.lastName);
}

3. Define toString() Behavior

In addition, we want to be able to easily visualize our sorted arrays. Therefore, we should define how our Employee class looks when converted to a string so we can easily call System.Console.log on it.

JS++ internal types use a “unified type system” where everything inherits from System.Object. If we look at the System.Object.toString documentation, we can see that System.Object.toString is a virtual method based on its signature:

public virtual string toString()

We override it with this code:

public override string toString() {
    return this.firstName + " " + this.lastName;
}

Thus, whenever we want a string representation of our Employee object, we will get the employee’s first name followed by his last name. This will help us visualize our sorted employees.

4. Instantiate some Employees

The next lines of code instantiate the Employee class and inserts them in an array:

Employee zig  = new Employee("Zig", "Ziglar");
Employee john = new Employee("John", "Smith");
Employee abe  = new Employee("Abe", "Lincoln");

Employee[] employees = [ zig, john, abe ];

Currently, the array is unsorted, and “Zig Ziglar” will be the first element.

5. Sort the Array

Sorting is as simple as one line of code:

employees.sort();

It’s just one line of code because we implemented IComparable<T>. Instead of implementing IComparable<T>, we could have also used the other overload of Array.sort, which expects a callback:

employees.sort(Comparison(Employee a, Employee b) {
    return a.lastName.compare(b.lastName);
});

The callback allows flexibility; for example, you may choose to sort by employee first name in some cases.

Implementing IComparable<T> simply provides a default sort so you can use System.Array.sort without a callback. These are the signatures for the System.Array.sort overloads:

public T[] sort() where T: IComparable<T>
public T[] sort(Comparison(T element1, T element2) comparator)

Thus, if you do not provide a callback, you are using the overload that expects a class implementing IComparable<T>. If you try to sort objects whose respective classes do not implement the IComparable interface, you’ll receive an error:

[  ERROR  ] JSPPE5056: System.Array.sort()' can only sort classes implementing 'IComparable'. Please implement 'IComparable' for `Employee' or use 'System.Array.sort(Comparison(T element1, T element2) comparator) at line 23 char 0 at test.js++

6. Print the Result

The final step is to just print the result:

Console.log(employees.join(", "));

Et voila!

(The toString method we implemented earlier will get called for each element that gets joined. Thus, you get a readable output.)

JS++ 0.9.0: Efficient Compile Time Analysis of Out-of-Bounds Errors

I promised a breakthrough for our next release.

We are proud to announce JS++ efficiently analyzes and prevents out-of-bounds errors. An out-of-bounds error occurs when you attempt to access a container element that doesn’t exist in the container. For example, if an array has only three elements, accessing the tenth element is a runtime error.

In C, you risk buffer overflows. In C++, you risk buffer overflows and exceptions. In Java and C#, you get an exception at runtime. If exceptions are uncaught, the application terminates. If segmentation faults occur, the application terminates. In the case of buffer overflows, you open your application to a variety of exploits.

As we will show, we can perform out-of-bounds analysis with only a ±1-2ms (milliseconds) overhead on complex projects. There is virtually no effect on compile times with our invention.

Out-of-bounds errors have plagued computer science and programming for decades. Detecting these errors at compile time has ranged from slow to impossible, depending on the language design. With that said, let’s first explore the problems which influenced the design.

Problems

Basic Cases to Handle

In all of the following cases, you cannot predict the value at compile time:

import System;

int[] arr = [ 1, 2, 3 ];

Console.log(arr[Math.random(1, 100)]);
Console.log(arr[getUserInput()]);
Console.log(arr[getValueFromFile()]);
Console.log(arr[API.getTweetLimit()]);

JS++ doesn’t stop at array indexes. Array indexes are limited to numeric values. What about arbitrary string keys on System.Dictionary<T>? Yes, we handle these too:

import System;

auto dict = new Dictionary<string>();

Console.log(dict[Math.random(1, 100).toString()]);
Console.log(dict[getUserInput()]);
Console.log(dict[getTextFromFile()]);
Console.log(dict[API.getTwitterUsername()]);

These are the basic cases. It gets more complex with branching logic:

import System;

Dictionary<string> dict = {
    "1":  "a",
    "10": "b"
};

bool yes() {
    return Math.random(0, 100) > 50;
}

if (yes()) {
    dict["20"] = "c";
}

string key = Math.random(0, 100).toString();
if (dict.contains(key)) {
    Console.log(dict[key]);
}
else {
    Console.log(dict[key + "0"]);
}

These are the very basic cases. There are more… a lot more. All the corner cases you need to explore are outside the scope of this announcement.

Compile Times Must Be Fast

Efficiency is the key. We can’t announce 30% faster compile times in the previous release and simultaneously promise a breakthrough that will cause compile times to explode exponentially.

Clearly, following every branch, virtual function call, external function call, and then some would not be a realistic proposal.

First, let’s look at a basic benchmark so we know what we’re comparing against. In the last release, 0.8.10, I measured “Hello World” compile times. With all of the analyses we added in 0.9.0 (the latest release), how much did it increase compile times for “Hello World”? A little under two (2) milliseconds:

Version Total Time
JS++ 0.8.10 72.6ms
JS++ 0.9.0 74.2ms
(Lower is better)

The test system is the exact same as the one we used to measure compile times for 0.8.10:

Intel Core i7-4790k
32gb DDR3 RAM
Samsung 960 EVO M.2 SSD
Debian Linux 9

However, “Hello World” is not a perfect benchmark. How long does it take to compile real-world projects with thousands of lines of code that make lots of array and dictionary accesses? Here are three projects before we introduced compile-time analysis of out-of-bounds errors:

Compile times for 0.8.10 – before out-of-bounds checking
Line Count Source Files Count Total Time
1,137 lines 27 files 124.8ms
4,210 lines 42 files 164.4ms
6,019 lines 72 files 224.6ms
(Lower is better)

Here are compile times after we introduced analysis of out-of-bounds errors:

Compile times for 0.9.0 – detection of out-of-bounds at compile time
Line Count Source Files Count Total Time
1,140 lines 27 files 124.4ms
4,148 lines 41 files 165.4ms
5,942 lines 71 files 224.2ms
(Lower is better)

There’s a slight change in line and file counts due to the inclusion of a ‘Base64’ library, which – during the 0.9.0 refactoring – I removed and replaced with the Standard Library’s System.Encoding.Base64. (The code is the exact same.)

The above projects include both frontend and backend code. They include lots of modules, classes, arrays, dictionaries, and other complexities. I’ve included source file counts to account for disk I/O.

It can be observed that there is virtually no performance penalty for dealing with out-of-bounds errors at compile time. The results are within ±1ms (milliseconds).

Nullable Types are a Problem

Expressing nullability is important in computer programming. For example, a file might have a creation date and last access time. For a new file, there may never have been a “last access time”; thus, it might be ideal to use a nullable data type in this case.

Nullable types are a solved problem in other languages. We considered having Array<T> return T?, but there would be issues with that as presented by Anton Rapetov, our lead compiler engineer:

int[] intArr = [ 1, 2 ];
int? intEl2 = intArr[2];
if (intEl2 == null) {
    Console.log("Definitely out of bounds");
}

int?[] nullIntArr = [ 1, null ];
int? nullIntEl2 = nullIntArr[2];
if (nullIntEl2 == null) {
    Console.log("Might be out of bounds, might just be an access of a null element");
}

Usability

Even if returning nullable types worked, there would be significant usability issues. For example, the following is common code:

int[] arr = [ 1, 2, 3 ];
for (int i = 0, len = arr.length; i < len; ++i) {
    arr[i]++;
    // or
    arr[i] += 1;
}

In the above code, it's clear an out-of-bounds access can never occur. Nonetheless, if an array access returns T?, type conversions would be necessary before the ++ or += 1 operations can occur so we aren't adding to a null value. We need a way to avoid making the user do this for common operations. In fact, for common operations, we want you to be able to write the code exactly as you would above.

Exceptions

In a statically-typed programming language, exceptions allow us to return T for an Array<T> without compromising correctness on an out-of-bounds access. For example, if we declare an Array<int>, a 'pop' method can only return a value of type int or throw an exception. If an exception is thrown, it is none of the concern of the type checker. At compile time, it would not be possible to determine whether or not the exception will be thrown. Yet, an uncaught exception will result in premature program termination at runtime.

Here's an example of how exceptions might be implemented for a container in JS++:

class Array<T>
{
    var data = [];

    T pop() {
        if (this.data.length > 0) {
            return this.data[this.data.length - 1];
        }
        else {
            throw new OutOfBoundsException("Array is empty.");
        }
    }
}

By using exceptions, we never sacrifice type checker performance and compile times. Bounds checking is still performed, but the dark side of exceptions is that it can terminate the application if uncaught.

If we avoid throwing exceptions, and just let JavaScript return undefined, we'd be walking into TypeScript territory and just letting our type system become unsound because it would be "practical." While you might convert undefined to int as zero, there aren't always sensible default values for all JS++ types (e.g. a callback type or a non-nullable class Foo with no default constructor). Speaking of default values...

Default Initialization

Facebook discovered a problem with C++ maps and default initialization in their code that can lead to bugs:

std::unordered_map<std::string, int> settings{};

// ...

std::cout << "Timeout: " << settings["timeout"] << std::endl;

In the above code, simply printing the value of "timeout" can cause it to be zero-initialized. This led us to conclude that default initialization of missing keys would not be a solution. Default initialization of a map of word counts to zero for missing words is innocuous, but an accidental initialization of timeout or price values to zero can lead to substantially different bug severities.

The Breakthrough: null vs undefined

We wanted to have nullable types in the language. We wanted programmers to be able to express the following:

class Person
{
    string firstName = "";
    string? middleName = null;
    string lastName = "";
}

As of the latest release (0.9.0), the above code will work because we've introduced nullable types.

However, I want to revisit an example on nullable types earlier. When we decided to move forward with nullable types, a suggestion was brought up to return T? from array accesses. This example was given as a counter-argument:

int[] intArr = [ 1, 2 ];
int? intEl2 = intArr[2];
if (intEl2 == null) {
    Console.log("Definitely out of bounds");
}

int?[] nullIntArr = [ 1, null ];
int? nullIntEl2 = nullIntArr[2];
if (nullIntEl2 == null) {
    Console.log("Might be out of bounds, might just be an access of a null element");
}

We keep a record of all our meetings. While we didn't explicitly discuss undefined at all, I was in a hurry and summarized our meeting as:

>>>> * There's a problem differentiating between 'null' and
>>>> 'undefined':
>>>>
>>>> ```
>>>> Foo?[] arr = [new Foo(), null];
>>>> auto el1 = arr[1];
>>>> auto el2 = arr[2];
>>>> ```
>>>>
>>>> el1 has type Foo?
>>>> el2 has type Foo?
>>>>
>>>> el1 has value null // el1 exists but is null
>>>> el2 has value null // el2 does NOT exist but is also null
>>>>
>>>> To deal with this, we can add a `hasIndex(int i)` method to
>>>> containers.

Subconsciously, this led to the realization that all we had to do was differentiate between null and undefined in our type system.

Introducing Existent Types

In JavaScript, null means that a value exists but is an "empty value," and undefined means no value exists at all. A basic example is here:

var x = null;
var y;

console.log(x); // null
console.log(y); // undefined

This illustrates the basic concept; unfortunately, JavaScript is inconsistent:

var x = null;
var y;
var z = undefined;

console.log(x); // null
console.log(y); // undefined
console.log(z); // undefined

JS++ has different semantics. First of all, in JS++, all variables must be initialized; therefore, you can't have a variable reference return undefined... ever. Secondly, null means "empty value," but undefined in JS++ means "out-of-bounds error."

JS++ introduces existent types, which uses the + syntax, to describe container accesses:

int[] arr = [ 7, 8, 9 ];

int+ x = arr[0];
int+ y = arr[1000];

We can think of existent types as the "bounds-checked type." I'm a big believer in simplicity. Rather than trying to calculate whether the container access is within-bounds or out-of-bounds at compile time, we delay this check to runtime via the code generator. Existent types are not purely a type checking innovation. The type provides guidance to the code generator to generate code such as the following:

int[] arr = [ 7, 8, 9 ];

int+ x = 0 < arr.length ? arr[0] : undefined;
int+ y = 1000 < arr.length ? arr[1000] : undefined;

We don't actually generate code this way, but it helps illustrate the concept for developers coming from backgrounds in C, C++, C#, Java, etc.

By default, int+ and int are not compatible types. I'll start by introducing the "safe default operator":

int[] arr = [ 7, 8, 9 ];

int+ x = arr[0];
int+ y = arr[1000];

int a = x ?? 0;
int b = y ?? 1;

The "safe default operator" will check if the left-hand side is undefined. If the value is undefined, the evaluated value of the right-hand side of the ?? operator is returned. Otherwise, the left-hand side is returned. In the case of the example above, 'a' will have the value of 7 because 'x' was within-bounds. 'b' will have the value of '1' because 'y' was out-of-bounds, and, thus, the alternative value provided to the ?? operator was used.

T+ cannot be the element type

The problem with JavaScript is that you can have an array of undefined values:

var arr = [ undefined, undefined, undefined ];

In the above case, JavaScript would not be able to differentiate between a within-bounds undefined and an out-of-bounds undefined. In JS++, an existent type cannot be the element type of an array or other container:

int+[] arr = []; // ERROR

[ ERROR ] JSPPE5204: Existent type `int+' cannot be used as the element type for arrays

Therefore, the invention of existent types cannot be retroactively applied to JavaScript.

If you want to represent an array element as having an "empty" value, you have to use nullable types...

Nullable Types + Existent Types

The following describes the basic syntax for the nullable and existent types being introduced in version 0.9.0:

int a = 1;  // 'int' only
int? b = 1; // 'int' or 'null'
int+ c = 1; // 'int' or 'undefined'

However, sometimes we want an array element to contain the "empty" value. In this case, we can combine nullable types with existent types using the following syntax:

int?+ d = 1; // 'int' or 'null' or 'undefined'

In this way, JS++ doesn't have the ambiguity of an undefined value that can be a within-bounds access and also an out-of-bounds access.

Usage with Dictionaries

Existent types can also be used with System.Dictionary<T>. We just introduced how nullable and existent types can be combined so let's use the combination:

import System;

Dictionary<bool?> inviteeDecisions = {
    "Roger": true,
    "Anton": true,
    "James": null, // James is undecided
    "Qin": false
};

bool?+ isJamesAttending = inviteeDecisions["James"]; // 'null'
bool?+ isBryceAttending = inviteeDecisions["Bryce"]; // 'undefined'

In the above code, we use the ?+ syntax to combine nullable and existent types. We're throwing a party, and we want to keep track of the decisions of our invitees. If the invitee's decision is true, he's coming to the party. If the invitee's decision is false, he won't be attending. If the invitee's decision is null, he is undecided. Finally, if the invitee's decision evaluates to undefined, he was not actually invited.

Naturally, the operators that apply to nullable types and existent types (such as the ?? safe default operator) will also apply to the combined ?+ type as well. Code will just be generated to check for null and undefined when using the combined ?+ type.

Beyond arrays and dictionaries, existent types can be applied to the other Standard Library containers (such as Stack<T> and Queue<T>) and even user-defined containers.

Safe Navigation Operator

Besides not being able to differentiate from a within-bounds undefined from an out-of-bounds undefined, JavaScript suffers from another problem:

var arr = [ 1 ];
console.log( arr[1000].toString() );
console.log( "This will never get logged." );

The above code will never reach line 3. The reason is because arr[1000] evaluated to undefined, and you can't call the toString() method on undefined so you'll get a runtime 'TypeError'. In JS++, this isn't a problem because the compiler will detect your attempt to use the . operator and suggest for you to use the ?. safe navigation operator instead:

import System;

int[] arr = [ 1 ];
Console.log( arr[1000].toString() );
Console.log( "This will eventually get logged." );

[ ERROR ] JSPPE5200: The '.' operator cannot be used for nullable and existent types (`int+'). Please use the '?.' safe navigation operator instead at line 4 char 13

If we refactor, we'll discover that, unlike the ?? safe default operator, ?. can return undefined and evaluates to an existent type T+:

import System;

int[] arr = [ 1 ];
Console.log( arr[1000]?.toString() );
Console.log( "This will eventually get logged." );

[ ERROR ] JSPPE5024: No overload for `System.Console.log' matching signature `System.Console.log(string+)' at line 4 char 0

So one possible fix is to provide a default value:

import System;

int[] arr = [ 1 ];
Console.log( arr[1000]?.toString() ?? "out of bounds" );
Console.log( "This will eventually get logged." );

It finally compiles, and we get the following output:

out of bounds
This will eventually get logged.

No crashes and no exceptions can occur.

Inspecting 'undefined'

Oftentimes, when you encounter an out-of-bounds error, you might want to skip to the next iteration over the container or return from a function. Essentially, you want to "skip" code that was written for within-bounds accesses. In JS++, it's as simple as comparing against the undefined value:

import System;

int[] arr = [ 1 ];

for (int i = 0; i < 10; ++i) {
    int+ element = arr[i];
    if (element == undefined) {
        continue;
    }

    int x = (int) element;

    Console.log(x + 1);
    Console.log(x + 2);
    Console.log(x + 3);
}

The C-style cast to int is safe because we already checked for and skipped out-of-bounds accesses. We can also use the safe default operator instead in the code above.

Finally, our output:

2
3
4

This allows us to elegantly write large chunks of code for within-bounds accesses while skipping, returning, or just ignoring out-of-bounds accesses. We can even log the out-of-bounds error to stderr by using System.Console.error.

Downloads

We're providing download links for the latest release (0.9.0) and the previous version (0.8.10). We want you to be able to verify our claims and benchmarks.

JS++ 0.9.0 (latest) – includes out-of-bounds checking
Platform Download Link
Windows Download (32- and 64-bit)
Mac OS X Download (32- and 64-bit)
Linux Download (32-bit)
Download (64-bit)

JS++ 0.8.10 – before out-of-bounds checking
Platform Download Link
Windows Download (32- and 64-bit)
Mac OS X Download (64-bit)
Linux Download (32-bit)
Download (64-bit)

What's Next?

Our first priority is to manage engineering complexity. We have to refactor our tests, and none of this will show up for you, the user. As I write this, I don't know what to expect. Existent types can bring demand for JS++, but we don't have the resources to manage this demand. Instead, we have to stay disciplined in sticking to our own internal schedules to ensure the long-term success of JS++. We listen to user input, but we don't (and can't) follow hype and trends. JS++ over the next 25 years will be more important than JS++ over the next 25 days. I point to Haskell as an example: it's a programming language that is well thought-out and has persisted for 29 years.

We have users that have followed us for years, and we thank all of them for giving us the motivation to persist. If you're willing to be patient and watch JS++ evolve, I urge you to join our email list. The sign-up form for our email list can be found by scrolling to the bottom of this page.

Final Words

Existent types were co-invented by me and Anton Rapetov (lead compiler engineer for JS++).

We solved compile-time analysis of out-of-bounds errors via traditional nominal typing. Thus, there is no performance difference for JS++ checking whether int can be assigned to string or whether int+ can be assigned to string. This explains the ± 1ms compile time difference for compile time out-of-bounds analysis.

We place heavy emphasis on compile times because we know long compile times hurt developer productivity.

When existent types are used correctly, you should never get premature or unexpected program termination.

There is a full tutorial on nullable and existent types available here.

JS++ 0.8.10: Faster Compile Times, Stacks/Queues, Unicode, Base64, and More

The next version of JS++ (not this one) will be a breakthrough. It is on par with ‘external’ in its importance to JS++. Stay with us, and stay tuned.

In this latest release of JS++, we’ve done a lot: we’ve improved on our compile times which already lead the competition by an order of magnitude (with room for more improvement), we’ve substantially expanded the Standard Library, made a UX improvement to generics (without breaking any existing code), and we’ve fixed a lot of bugs (which are mostly minor at this point after years of engineering).

Faster Compile Times

import System;

Console.log("Hello World");

On the Core i7-4790k:

Version Total Time
JS++ 0.8.5 96.2ms
JS++ 0.8.10 72.6ms
(Lower is better)

System Specifications:

  • Intel Core i7-4790k
  • 32gb DDR3 RAM
  • Samsung 960 EVO M.2 SSD

As evidenced in the table above, the latest version is now compiling “Hello World” 32.51% faster. There is still room for more improvement to compile times, but it is not our current priority. We improved compile times in this release by pre-parsing and caching the JS++ Standard Library. There is more to Standard Library compilation than this single step, but we wanted to address this problem as it was the largest performance regression in our profiling. Practically, this allows us to substantially expand the JS++ Standard Library with an O(log N) cost to compile times versus the previous O(n) cost of adding new libraries.

And because we’re able to add substantially more libraries… we’ve done exactly that.

System.Stack<T> and System.Queue<T>

JS++ now provides stack (LIFO) and queue (FIFO) data structures. Stacks are an abstraction over JavaScript arrays and are very fast.

import System;

auto stack = new Stack<int>();
stack.push(1);
stack.push(2);
Console.log(stack.pop()); // 2

In the generated code, the ‘push’ call is a direct ‘push’ call on the internal array representation behind the stack so there is no performance loss in using the stack abstraction instead of arrays.

Next, we also have queues via System.Queue. It’s not as simple as an abstraction over JavaScript arrays like stacks because re-indexing an array would be an O(n) operation. Instead, to guarantee O(1) pop operations, we use a ring buffer. (Credit goes to Anton, our lead engineer.)

import System;

auto queue = new Queue<int>();
queue.push(1);
queue.push(2);
Console.log(queue.pop()); // 1

At a micro-optimization level, you might question our decision on the ring buffer backing store. However, computer science is important. If you take a look at my JS++ stringset library, you’ll see this benchmark over a dictionary of ~49,000 terms:

StringSet : 70ms     (00.07 seconds)
string[]  : 82299ms  (82.29 seconds)

Clearly, the performance difference here is substantial, but it should be no surprise to anyone that understands data structures: an array has O(n) lookups, and a set has O(1) lookups.

Documentation:

System.Encoding

This module introduces a lot of useful new features, but I’m going to break it down.

Base64 Encoding and Decoding

A common operation in web development is Base64 encoding and decoding. For example, the HBase REST API requires Base64 encoding/decoding. You can also use Base64 to encode binary data and files as ASCII, such as converting canvas image data into data URIs.

Wouldn’t it be nice to have all of this functionality readily available in the language you use?

import System;
import System.Encoding;

string quote = "Man is distinguished, not only by his reason, but by this singular passion from other animals, which is a lust of the mind, that by a perseverance of delight in the continued and indefatigable generation of knowledge, exceeds the short vehemence of any carnal pleasure.";

string encoded = Base64.encode(quote);
Console.log(encoded);

string decoded = Base64.decode(encoded);
Console.log(decoded);

The more you master JS++ and know how to use the Standard Library, the more efficiently you can get work done compared to JavaScript.

Documentation: System.Encoding.Base64

UTF-8, UTF-16, and UTF-32

Dealing with Unicode is a key aspect of writing world-ready software. UTF-8, UTF-16, and UTF-32 are encoding schemes defined in the Unicode Standard, and JS++ now supports encoding and decoding of all of these in the System.Encoding module.

Here’s an example of UTF-8 encoding:

import System;
import System.Encoding;

byte[] encoded = UTF8.encode("€");

string toHex = "";
foreach(byte b in encoded) {
    toHex += "\\x" + b.toHex().toUpperCase();
}

Console.log(toHex); // "\xE2\x82\xAC"

We just want to take a moment to remind you the importance of the JS++ type system in scenarios like this and how it naturally interoperates with so many areas of computing. In this case, the ‘byte’ data type is a natural fit for dealing with standard Unicode encoding schemes.

Documentation:

URI Encoding and Decoding

This module provides the ECMAScript 3 encodeURI, encodeURIComponent, decodeURI, and decodeURIComponent functions.

Documentation: System.Encoding.URI

Superior Documentation

Documentation is one of the strengths of JS++. We have over 600+ pages of handwritten documentation.

With the release of the System.Encoding.URI module, we wanted to expand on this. Have you ever wondered about the difference between encodeURI and encodeURIComponent in JavaScript? This unfortunate naming scheme, dating back to ECMAScript 3, is a source of confusion. Developers often cite the Mozilla Developer Network (MDN) for documentation, but their explanation is equally as confusing as the naming scheme and lacks useful information:

encodeURIComponent

Stack Overflow wasn’t tremendously more helpful either, and most answers once again just go over which characters get converted and which do not. This doesn’t help the practicing developer learn or memorize which function to use.

Fortunately, JS++ has you covered. We explain the difference between encodeURI and encodeURIComponent clearly, and we also provide an explanation of best practices to help you navigate the confusion:

https://docs.onux.com/en-US/Developers/JavaScript-PP/Language-Guide/encodeuri-vs-encodeuricomponent

Improving Generic Programming UX: Default Constraint

We’ve improved the user experience (UX) for generic programming in JS++.

Previously, the default constraint for generic type parameters was System.Object. However, for performance reasons, the default constraint did not allow primitive types. For example:

class Foo<T> // same as 'Foo<T: System.Object>'
{
}

auto foo = new Foo<string>(); // ERROR (previously)

Instead, you needed to specify the “wildcard constraint” (see the docs) if you wanted to allow primitive types as arguments, as System.Array and System.Dictionary do.

However, this was not the most useful default. All your old code will still work, but we just made the defaults more useful. Starting from version 0.8.10, all your generic classes will have the wildcard constraint as the default. You can declare a plain generic class and instantiate it with primitive types as type arguments.

‘this’ Semantics

First and foremost, before we announce this change, we have to say all semantics relating to this are subject to change.

We’ve been well aware of how JavaScript’s this rules differ from the semantics that users from other programming languages are familiar with. However, previous versions of JS++ raised a ‘0000’ (unimplemented) error here, and the fix was non-obvious:

class Foo
{
    string message = "Test";
    void bar() {
        $(document).click(void() {
            $("#button").text(this.message); // JSPPE0000 error
        });
    }
}

You had to manually capture the ‘this’ value for closures inside classes:

class Foo
{
    string message = "Test";
    void bar() {
        Foo _this = this;
        $(document).click(void() {
            $("#button").text(_this.message); // OK
        });
    }
}

First of all, I apologize for the cryptic error message. We work very hard to make sure our error messages are easy to understand, and we always try to suggest the fix in the error message itself where possible. In this case, we always thought the ‘this’ semantics would be settled within a reasonable time and thought the “unimplemented” error would only be temporary. We were wrong. Thus, it’s important to know we may change the rules and semantics again prior to JS++ 1.0.

In the current release, the this keyword – when used inside classes – refers to the class instance by default. There are cases where you might want it to retain JavaScript this semantics, and, at least for closures inside classes declared with the function return type, you can cast it to external:

class Foo
{
    string message = "Test";
    void attachEvents() {
        $("#button").click(function() {
            var $this = (external) this;
            $this.text(this.message); // OK
        });
    }
}

We are open to input on this functionality.

You can review our plans on the this documentation page under the header, “‘this’ Casting inside Classes”.

Bug Fixes

  • Disable auto creation of the arguments object
  • File extensions are case-insensitive
  • Fix crash during access to a property with undeclared type (reported by @lorveg in JS++ chat)
  • Fix contravariance check in foreach loop
  • Incorrect scoping and code generation for ‘external’
  • Segfault for interface with generic variants
  • Fix missing class name for inheritance in error message

Tips & Tricks: Overriding ‘toString’

JS++ has a default ‘toString’ method implementation but, sometimes, it is necessary to override this implementation. For example, when using Console.log, it may be desirable to be able to fully log and inspect a complex JS++ object.

In addition to the Unified External Type, there is also a “Unified Internal Type”: System.Object. All JS++ classes, including user-defined classes, inherit from System.Object. Due to auto-boxing, even primitive types such as int (wrapped by System.Integer32), inherit from System.Object.

Aside: Don’t worry about the performance implications of auto-boxing. JS++ is able to optimize auto-boxing to the point that toString is actually 7.2% faster in JS++ than JavaScript in the worst case (assuming the JavaScript variable is monomorphically-typed) and more than 50% faster for polymorphically-typed (and potentially type-unsafe) JavaScript variables as shown in benchmarks here.

System.Object has a toString method which is marked as virtual. In other words, this method can be overridden by derived classes – which are effectively all classes in JS++. Here’s an example of how to do it:

import System;

class Point
{
    int x;
    int y;

    Point(int x, int y) {
        this.x = x;
        this.y = y;
    }

    override string toString() {
        return "(" + x.toString() + ", " + y.toString() + ")";
    }
}

Point p = new Point(1,2);
Console.log(p); // "(1, 2)"

You’ll notice the Console.log statement doesn’t even make an explicit toString call. The reason is because passing any JS++ object to Console.log will call the toString method on the object for you.

Join the JS++ Chat Room

If you need instant help or just want to talk about JS++, feel free to join us in the JS++ Chat Room:

https://chat.onux.io/signup_user_complete/?id=6eoxd4erotgwdgin8p6x9uupyc

The special thing about this chat room is that our internal team uses this chat software to collaborate. While we are mostly in private channels, we are almost always available in the public chat rooms. If you need help, you can get help instantly from the creators of JS++.

Tips & Tricks: fromExternal/toExternal Design Pattern

JS++ provides toString and fromString (one example) methods in the Standard Library. However, it can be argued that the external type is just as important or even more important in JS++ as string.

We introduce a design pattern for converting complex user-defined JS++ types (such as classes) to JavaScript.

toExternal

You can define a toExternal method that enables you to convert an object of an internal type to external:

import System;

class Point
{
    int x;
    int y;

    Point(int x, int y) {
        this.x = x;
        this.y = y;
    }

    function toExternal() {
        return {
            x: x,
            y: y
        };
    }
}

Point p = new Point(2, 3);
var p2 = p.toExternal(); // conversion to 'external'
Console.log(p2.x); // 2
Console.log(p2.y); // 3

fromExternal

Likewise, you can convert incoming JavaScript data to a complex, user-defined JS++ type:

import System;
import System.Assert;

class Point
{
    int x;
    int y;

    Point(int x, int y) {
        this.x = x;
        this.y = y;
    }

    static Point fromExternal(obj) {
        assert(typeof obj == "object", "Expected external 'object' type");
        assert(typeof obj.x == "number", "Expected incoming external to have numeric 'x' property");
        assert(typeof obj.y == "number", "Expected incoming external to have numeric 'y' property");
        return new Point(Integer32.fromString(obj.x), Integer32.fromString(obj.y));
    }
}

Point p1 = Point.fromExternal({ x: 2, y: 3 });
// Point p2 = Point.fromExternal({ x: "x", y: 3 }); // this will fail
// Point p3 = Point.fromExternal({ x: 2, y: "y" }); // this will fail

Protecting References

For functions, you don’t want to send out a direct reference to external JavaScript. Otherwise, external JavaScript code can modify the JS++ reference in unsafe ways. Therefore, you should wrap the function using closures:

class Foo
{
    void bar() {}

    function toExternal() {
        Foo self = this;

        return {
            bar: void() {
                self.bar();
            }
        };
    }
}

Furthermore, you can use the Standard Library’s System.BoxedExternal to handle this case without wrapping in a closure:

import System;

class Foo
{
    BoxedExternal bar;

    Foo() {
        this.bar = new BoxedExternal(void() {
        // ...
        });
    }
 
    function toExternal() {
        return {
            bar: this.bar
        };
    }
}

If the reference to the function accidentally escapes to external, you’ll be alerted by the compiler:

[ ERROR ] JSPPE5000: Cannot convert `System.Dictionary‘ to `external’ at line 14 char 15

However, if you actually intended to allow the function reference to escape to external, you can call the unbox method on System.BoxedExternal:

import System;

class Foo
{
    BoxedExternal bar;

    Foo() {
        this.bar = new BoxedExternal(void() {
            // ...
        });
    }
 
    function toExternal() {
        return {
            bar: this.bar.unbox()
        };
    }
}

The above code will now compile and the bar function can be passed to external code. However, unlike the code where we wrapped the function in a closure, the external code can now modify the reference to the bar function directly so be careful.

For arrays and containers, you can likewise pass a shallow copy or manually clone each element – depending on the level of trust and safety you desire.

Tips & Tricks: Only Fields are ‘private’ by Default

Programmers often complain about the verbosity of Java. Once you specify all the modifiers that must be applied, it’s not difficult to see how it can quickly become verbose:

public static void veryLongNamingConventions() {
    // ...
}

JS++ does this differently. Following the OOP principle of encapsulation, JS++ provides convenient default rules for access modifiers.

By default, only fields (variable members of classes) are private. All other class members – such as methods, getters, setters, and constructors – are public by default.

This makes it very easy to write concise code:

class Point
{
    int x, y;

    Point(int x, int y) {
        this.x = x;
        this.y = y;
    }

    int getX() { return this.x; }
    int getY() { return this.y; }
}

In the above code, the fields x and y are private. Meanwhile, the constructor and the getX/getY methods are all public. We can be explicit and manually specify the access modifiers, but it’s not necessary in JS++.

JS++ 0.8.5: Bug Fix Release

This is a small bug fix release.

The following minor issues have been fixed:

  1. Dictionary expression of empty arrays now has the correct type inferred
  2. Labels for foreach statements
  3. Type inference for System.Array.reduce
  4. Function inlining for System.String methods: padRight(int), padLeft(int), startsWith, endsWith

Issue #1:

Dictionary<unsigned int[]> x = {
    "foo": (unsigned int[]) [],
    "bar": (unsigned int[]) []
};

The casts were previously required in 0.8.4, but, in the latest release, casting of empty arrays is no longer necessary.

Issue #2:

outerLoop: foreach (var row in json) {
    // ...

    foreach(var cell in row.Cell) {
        // ...
        continue outerLoop;
    }
}

Issue #3:

string[] keys = dict.keys();
int x = keys.reduce(
    int(string previous /* type of 'previous' param */, string current) {
    },
    0 // and this type
);

In the above case, the two types must match. However, the type checker was inferring ‘external’ instead. This has been fixed in the latest release.

JS++ 0.8.4: Advanced Generics and System.String Expansion

We have significantly expanded the Standard Library with this release. In particular, System.String has undergone significant expansion.

System.String Highlights

between

string quotedWords = '"duck" "swan" "crab"';
// 'between' is smart enough to allow the same string to be used as a start and end delimiter
string[] words = quotedWords.between('"', '"');
Console.log(words); // [ "duck", "swan", "crab" ]

Documentation page: click here

format: C-like printf

"%s is %d years old".format("Joe", 10) // Joe is 10 years old

Documentation page: click here

escape

"a\r\nb".escape() // a\\r\\nb

Documentation page: click here

truncate

string text = "The quick brown fox jumped over the lazy dog.";
 
Console.log(text.truncate(9)); // "The quick..."

Documentation page: click here

repeat

"*".repeat(5) // *****

Documentation page: click here

count

"foobar".count("foo") // 1
"FOOBAR".icount("foo") // 1

Documentation (count): click here
Documentation (icount): click here

contains

"abc".contains("b") // true
"ABC".icontains("b") // true

Documentation (contains): click here
Documentation (icontains): click here

New System.String Methods

Here are all the new methods available for strings in JS++:

  • between – Gets substrings between two delimiters (does not use regex)
  • compact – Removes whitespace globally
  • contains/icontains
  • count/icount
  • countLines
  • countNonEmptyLines
  • startsWith/endsWith
  • escape/unescape – Escape the escape sequence characters (e.g. \n -> \\n)
  • escapeQuotes/unescapeQuotes
  • format – Similar to C’s printf
  • insert/append/prepend
  • isEmpty – uses .length === 0 rather than str1 === “” for performance, not everyone has time to benchmark every detail
  • isLowerCase
  • isUpperCase
  • isWhitespace
  • joinLines – collapses a string composed of multiple lines into a single line
  • joinNonEmptyLines
  • padLeft/padRight – remember the NPM debacle?
  • quote/quoteSingle – wraps the string in quotes
  • unquote – removes quote pairs
  • repeat – “*”.repeat(3) == “***”
  • reverse
  • splitLines – splits a string into a string[] (array) based on newlines
  • trim, trimLeft, trimRight, trimMulti, trimMultiLeft, trimMultiRight
  • truncate – Cuts off the string at the specified length (with support for custom ellipsis)

There are close to 50 new string methods (48 including overloads, 39 otherwise), and these methods should cover most application-level usages. With documentation, this resulted in +1400 new lines of code to System.String. I’m happy to announce we actually still have more methods (for System.String and others) on the way.

Every single method is documented. All documentation is online and available at the System.String index page.

We avoided regular expressions as much as possible to avoid runtime FSM construction, which takes time and space. Therefore, prefer JS++ methods such as "abc".endsWith("c") over the traditional regex/JavaScript /c$/.test("abc").

The best thing about JS++ is that it’s a compiled language. This gives you performance benefits that a JavaScript library with string utilities can never give you. For example:

if ("abc".isEmpty());

becomes:

if ("abc".length===0);

and

"abc".quote()

becomes:

'"'+"abc"+'"'

The astute observer will notice that both the above methods can be further optimized to reach “perfect” optimization. However, there is no optimizing compiler inside JS++ yet, and inserting branching logic into the code generator will result in technical debt.

Our goal with the Standard Library is to make it easier than ever to write applications compared to JavaScript. Side effects of our work on the JS++ Standard Library are performance, size, and correctness. JS++ dead code elimination means we can add hundreds of methods to System.String, but you only pay for the methods you actually use. For performance, not every team can afford to hire a JavaScript performance expert. Even if you have the performance expert, he can’t be expected to micro-optimize and benchmark every method.

Finally, with the JS++ Standard Library, we can fully avoid the NPM left-pad debacle.

import System;

Console.log("1".padLeft(4, "0")); // "0001"

fromString

Previously, to convert a string to number in JS++, it was a little unintuitive. For example:

int x = +"1000"; // use the unary + operator

For all numeric types, we’ve introduced the fromString, fromStringOr, and fromStringOrThrow static methods. The above example can be re-written to use Integer32.fromString:

int x = Integer32.fromString("1000");

Advanced Generics

JS++ 0.8.4 introduces covariant and contravariant generic types (including upcasting and downcasting for types with variants). Covariance and contravariance are based on use-site variance. At this time, we are not introducing declaration-site variance at all; we have higher priorities. In addition, we’ve introduced generic constraints (subtype constraints, multiple constraints, wildcard constraints, and more).

Finally, we have support for generic functions and generic static methods.

Everything from basic to advanced generic programming in JS++ is covered in our generic programming documentation.

When we released version 0.8.0, we introduced only basic generics. In today’s 0.8.4 release, you can consider generics fully implemented.

I highly encourage reading the generic programming documentation. To put it all together, here’s generic covariance and contravariance together with use-site variance:

import System;
 
abstract class Animal {}
class Tiger : Animal {}
 
abstract class Pet : Animal {}
class Dog : Pet {}
class Cat : Pet {}
 
class PetCollection
{
    Pet[] data = [];
 
    void insert(descend Pet[] pets) {
        foreach(Pet pet in pets) {
            this.data.push(pet);
        }
    }
 
    ascend Pet[] get() {
        return this.data;
    }
}
 
auto myPets = new PetCollection();
 
// Write operations (descend, covariance)
myPets.insert([ new Dog, new Cat ]);
// myPets.insert([ new Tiger ]); // not allowed
 
// Read operations (ascend, contravariance)
Pet[] getPets = [];
Animal[] getAnimals = [];
ascend Pet[] tmp = myPets.get(); // read here
foreach(Pet pet in tmp) { // but we still need to put them back into our "result" arrays
    getPets.push(pet);
    getAnimals.push(pet);
}
 
// Now we can modify the arrays we read into above
getPets.push(new Dog);
getAnimals.push(new Dog);
getAnimals.push(new Tiger);
// getPets.push(new Tiger); // ERROR

Other Changes

  • Fix return types for System.String.charAt and System.String.charCodeAt
  • Fix type promotion to ‘double’. We now handle this better than languages like Java and C#. Thanks to our lead engineer, Anton, for the idea.
  • isEven() and isOdd(). You might think this is fizz buzz, but if you’re using the modulus operator, it’ll be slower. We use bitwise operations, and you might be interested in reading this article on how we took advantage of overflow behavior to improve performance while preserving correctness.
  • Fix System.Array.map and System.Array.reduce to support wildcard generic types.
  • Type inference of generic parameters for function calls. This is needed for System.Array.map and System.Array.reduce, but it’s also available for user-side code.
  • Fix System.Console.error when no console is available
  • Fixed error message with incorrect type for setters defined with no accompanying getters.
  • Fixed private access modifier for modules in a multi-file setting.
  • Fix callback types as generic arguments
  • Fix enum bitwise operations to reduce explicit casting

Leveraging Integer Overflow to Improve Performance While Preserving Correctness

If you use the upcoming UInteger32.isEven or isOdd methods, you’ll notice that it uses a bitwise AND operation. The reason, as described in a previous post, is because it improves performance.

However, while this is straightforward for all other integer wrapper classes, UInteger32 is an exception. According to ECMAScript 3 11.10:

The production A : A @B, where @ is one of the bitwise operators in the productions above, is evaluated as follows:

  1. Evaluate A.
  2. Call GetValue(Result(1)).
  3. Evaluate B.
  4. Call GetValue(Result(3)).
  5. Call ToInt32(Result(2)).
  6. Call ToInt32(Result(4)).
  7. Apply the bitwise operator @ to Result(5) and Result(6). The result is a signed 32 bit integer.
  8. Return Result(7).

The operands (and, thus, the result type) for the bitwise AND operation in ECMAScript are converted to 32-bit signed integers. System.UInteger32 represents an unsigned 32-bit integer.

This is inconvenient because we’d obviously have to fall back to the slower modulus operation for isEven/isOdd on UInteger32. Unless…

((Math.pow(2, 32) + 1) >>> 0 | 0) & 1 // true
((Math.pow(2, 32) + 2) >>> 0 | 0) & 1 // false

We can take advantage of overflow behavior. (Note: since we’re able to get the correct result by leveraging overflow behavior, we actually don’t perform the extraneous zero-fill right shift as illustrated in the example.)

This is completely safe because:

A) Mathematically, in base 2, the last bit will always be 1 for odd numbers and 0 for even numbers… no matter how big the number is.

B) Bitwise AND will compare both bits in equal-length binary forms. Thus, no matter how big the number is, when you AND against 1, it will always be 0000001 (or zero-padded to whatever length is needed). Therefore, all the preceding bits don’t matter, because they will always be ANDed against a zero bit. The only bit that matters is the trailing bit, see A for why this will always work.