Anton Lavrenov

Case study: Design Editor for Video Streams

Sun, 26 Apr 2020 00:00:00 GMT

The post describes steps to build a video editor prototype for organizing video streaming from the browser.

The idea

Do you know that you can create a video stream from the browser? Thanks to captureStream API we can generate a video stream from the canvas element and send it to the backend (and later send to other users). And we can draw anything on that canvas!

I am going to describe key points to make such an editor.

Open final demo. Looking for help with a similar application? Contact me.

How to start screen and video recording?

We will need two main API here: getUserMedia and getDisplayMedia.

var constraints = { video: true, audio: false };
let userPromise;
function getUserMedia() {
  if (!userPromise) {
    userPromise = navigator.mediaDevices.getUserMedia(constraints).catch(function (err) {
      console.log(err.name + ': ' + err.message);
    }); // always check for errors at the end.
  }
  return userPromise;
}

let displayPromise;
function getDisplayMedia() {
  if (!displayPromise) {
    displayPromise = navigator.mediaDevices.getDisplayMedia(constraints).catch(function (err) {
      console.log(err.name + ': ' + err.message);
    }); // always check for errors at the end.
  }
  return displayPromise;
}

Also I am going to use React to display video on the screen. Using functions above we can write simple hooks:

export const useUserStream = () => {
  const [stream, setStream] = React.useState(null);

  React.useEffect(() => {
    getUserMedia().then(function (mediaStream) {
      setStream(mediaStream);
    });
  }, []);

  return stream;
};

export const useDisplayStream = () => {
  const [stream, setStream] = React.useState(null);

  React.useEffect(() => {
    getDisplayMedia().then(function (mediaStream) {
      setStream(mediaStream);
    });
  }, []);

  return stream;
};

How to display video on the canvas?

I am going to use React and Konva framework to build a canvas application. Following Konva Video Tutorial I am going to write a react component to draw a video on a canvas:

// react hook to create a video DOM element from getUserMedia or getDisplayMedia stream
const useVideo = (stream) => {
  const video = React.useMemo(() => {
    return document.createElement('video');
  }, [stream]);

  React.useEffect(() => {
    if (!stream) {
      return;
    }
    video.srcObject = stream;
    video.onloadedmetadata = function (e) {
      video.play();
    };
  }, [stream]);
  return video;
};

const Video = ({ element }) => {
  const stream = element.type === 'user-video' ? useUserStream() : useDisplayStream();
  const video = useVideo(stream);

  const shapeRef = React.useRef();

  React.useEffect(() => {
    const anim = new Konva.Animation(() => {
      // do nothing in animation, it will just automatically redraw a layer
    }, [shapeRef.current.getLayer()]);
    anim.start();
    return () => anim.stop();
  }, [video]);

  return (
    <Image
      elementId={element.id}
      ref={shapeRef}
      draggable
      image={video}
      width={element.width}
      height={element.height}
      x={element.x}
      y={element.y}
      rotation={element.rotation}
    />
  );
};

To make the final demo I am going to use https://polotno.dev/ framework. It already has support for Drag&drop Feature and Transforming ability from the Konva.

How to capture canvas?

In Konva framework we have a node structure like this:

- Stage
 - Layer
  - Shapes

Stage represents just a div wrapper. And Layer is an actual canvas element on the screen. Using we can access canvas element in order to create a stream from it:

const App = () => {
  const layerRef = React.useRef();
  React.useEffect(() => {
    // on mount
    const layerNode = layerRef.current;
    // get canvas element from the layer
    const canvasElement = layer.getCanvas()._canvas;
    // create a stream from the canvas
    var stream = canvasElement.captureStream(25);
  }, []);
};

How to optimize the performance?

While making an application I noticed that it consume a lot of CPU. Yeah, video processing is expensive. But can we do anything to improve the performance? On my machine that app doesn’t work slow. It is faster. But hight CPU usage means high energy consumption (fast battery drain and hot temperature in my room).

That is CPU level before any optimizations:

The main reason is in video rendering on the canvas and animation hook:

React.useEffect(() => {
  const anim = new Konva.Animation(() => {
    // do nothing in animation, it will just automatically redraw a layer
  }, [shapeRef.current.getLayer()]);
  anim.start();
  return () => anim.stop();
}, [video]);

Konva.Animation is working with requestAnimationFrame it means it is trying to draw frames as fast as possible. Ideally, it is about 60 FPS. But do we need that large FPS for streaming? Probably your screen/webcam has lower FPS. Also a sending a stream into the backend with 60 FPS doesn’t make much sense. What if 30 FPS can do the work? What if a user on low-end device and he wants to reduce CPU usage even more with a lower stream quality?

In that case I am going to control FPS manually with different quality option:

// let's think we can have three levels of quality:
const INTERVALS = {
  high: 1000 / 50,
  good: 1000 / 30,
  low: 1000 / 5,
};

React.useEffect(() => {
  const interval = setInterval(() => {
    shapeRef.current.getLayer().batchDraw();
  }, INTERVALS[quality]);
  return () => clearInterval(interval);
}, [quality, video]);

CPU usage on my computer on a lower quality:

The result

Note: as the demo bellow is displayed in iframe, it may not fully work on some browsers. Try to open it in a new tab.

Looking for help with a similar application? Contact me.

Case study: Image Labeling with JavaScript

Fri, 20 Sep 2019 00:00:00 GMT

Need to label images to train a neural network?

This post will describe how to design a system for labeling objects on an image by a human.

Note: this post is about manual step, when we need to highlight objects BEFORE training a neural network.

I will show you how to use React, 2d canvas and Konva framework to make an app that allows you to highlight objects. Also, I will cover some edge cases specific to that task.

Open demo. Looking for help with a similar application? Contact me.

That prototype was built special for Labelbox.

The idea

In many areas of machine learning and AI we need to label images with objects to train the system. The action should be done by a human (who already know how to detect and label images, right?). So the human needs to draw something like “where is the tree here?”, “where is the house here?” etc.

So at first, the task is trivial. We need to make a system that allows us to see objects and draw some shapes on top of it with the free drawing:

So it is just a canvas with an image in it and a set of lines, that we can create with mouse events:

<Stage onMouseDown={startDrawing} onMouseMove={addNewPoints} onMouseUp={finishDrawing}>
  <BaseImage />
  <Regions regions={createdLines} />
</Stage>

Requirement - transparent, non-overlapping regions

To make the basic image visible, we need to make our regions transparent, so even after some drawing, you see the whole image. With the Konva and 2d canvas API, it is simple, we can just set alpha channel with opacity attribute:

<Line points={region.points} fill={region.color} opacity={0.7} closed />

That works just fine. You will see the underlying image. But it produces another issue, we can have overlapping regions:

In that case, we have overlapping road and car annotations. By the requirements of machine learning algorithms, such situations are not acceptable. Every pixel of an image should represent a color of particular annotation. In the case of overlapping objects, we have mixed colors. How to resolve the issue?

Solution 1 - use transparent layer

In Konva we can have several layers. Each layer has its own <canvas> element. We can style that canvas element manually with CSS. We will not use canvas API to set opacity, but instead, we will make the canvas transparent:

const Regions = ({ regions }) => {
  const layerRef = React.useRef(null);

  React.useEffect(() => {
    // find canvas element
    // warn: to use private API of Konva
    const canvas = layerRef.current.getCanvas()._canvas;
    // set opacity manually:
    canvas.style.opacity = 0.8;
  });

  return (
    <Layer ref={layerRef}>
      {regions.map((region) => {
        return <Line points={region.points} fill={region.color} closed />;
      })}
    </Layer>
  );
};

Now a region is not transparent when we are drawing it into the canvas element. But the canvas element itself is transparent, so the regions look like transparent. We don’t have the “overlapping sections” issues:

That solves the problem. But later in time, we have another requirement, we need the ability to “select” a region. When a region is selected it should have no transparency (or very tiny). But we can’t do that with the current approach. Why? Good question. Because when we set opacity with CSS we are setting a maximum opacity for all regions. How to fix it?

Solution 2 - cut regions from each other

The idea is simple. Let think that we have two overlapping regions A and B. First we are drawing the region A on canvas. Then we are erasing a part that overlaps with B. Then we draw the region B. Thanks to globalCompositeOperation from 2d canvas API, we can easily do it:

const Regions = ({ regions, selectedId }) => {
  return (
    <Layer>
      {regions.map((region) => {
        const isSelected = region.id === selectedId;
        return (
          <React.Fragment>
            {/* first we need to erase previous drawings */}
            {/* we can do it with  destination-out blend mode */}
            <Line
              globalCompositeOperation="destination-out"
              points={region.points}
              fill="black"
              closed
            />
            {/* then we just draw new region */}
            <Line
              points={region.points}
              fill={region.color}
              closed
              opacity={isSelected ? 1 : 0.8}
            />
          </React.Fragment>
        );
      })}
    </Layer>
  );
};

In this example I selected the giraffe. It is not transparent. Two zebras are transparent and they have overlapping regions, but we don’t see them, because we used a “cut” method:

Changing the brightness of the image

Sometimes images are too light or too dark. So we need the ability to change their brightness. We can easily do that with Brighten filter from Konva framework:

const BaseImage = ({ brightness }) => {
  const [image] = useImage(IMAGE_URL, 'Anonymous');

  const imageRef = React.useRef(null);

  React.useEffect(() => {
    if (!image) {
      return;
    }
    // we need to cache image in order to use filters
    imageRef.current.cache();
  }, [image]);

  return (
    <Layer>
      <Image
        image={image}
        ref={imageRef}
        filters={[Konva.Filters.Brighten]}
        brightness={brightness}
      />
    </Layer>
  );
};

And that approach works well for many cases.

But sometimes images are HUGE. They have a lot of pixels. Konva filters work by manipulating every pixel with JavaScript, it works on CPU level. But we can use the power of CSS filters. In many browsers (probably all) CSS filters work on GPU level. So it can be much faster:

const BaseImage = ({ brightness }) => {
  const [image] = useImage(IMAGE_URL, 'Anonymous');

  const layerRef = React.useRef(null);

  React.useEffect(() => {
    const canvas = layerRef.current.getCanvas()._canvas;
    // apply css filters manully
    canvas.style.filter = `brightness(${(brightness + 1) * 100}%)`;
  }, [brightness]);

  return (
    <Layer ref={layerRef}>
      <Image image={image} />
    </Layer>
  );
};

That approach works MUCH faster, even for very large images.

The result

After some polishing, implementing better navigation, showing the list of current labels we have a working prototype. The real client’s app is working a bit differently because it has some other extra features, but it was important to make a proof-of-concept to make sure that selected technologies will do the work.

With that system we can easily export mask images into machine learning algorithms.

Here is the full demo (or open it in a new tab):

Looking for help with a similar application? Contact me.

Case study: Seats Reservation Widget with Javascript

Tue, 23 Jul 2019 00:00:00 GMT

Do you want to make a seat booking platform on your website?

This post will describe steps to design a seat reservation system for music festival website made with JavaScript.

It will show how to use React, 2d canvas and Konva framework to make high-performance widget.

Open demo. Looking for help with a similar application? Contact me.

The idea

The client need the ability to reserve and book seats to the music festival directly on the website. It is important to see available sections, seats and their prices.

For a while the client was using https://www.seats.io/ to render seats chart and to allow selecting and booking. But as prices were coming up and there were some issues with backend integration and making sure that there is no double booking, the client decided to make its own solution to inject into the main website.

So the task was to create a similar solution with some simplifications that allow to draw charts like seats.io:

There is no business value in fancy animations, cool graphics, etc. I need to make a fast and simple tool for reservations.

Bootstrapping

As the client is already using React for the main application I should allow simple API and make a React component, so it is simple to use it in the workflow of the current codebase.

When you are making a chart or some data visualizations there are tons of available options and approaches.

You can create a chart with regular DOM (a bunch of divs), SVG element, 2d canvas or WebGL. As the client comes to me from the Konva website he already decided to make it with 2d canvas. But before jumping to consulting I was thinking about other approaches.

WebGL can be an overkill for the solution because it may be too low-level and requires implementation that hard to make in a short time and make it simple.

Bunch of divs and spans and SVG drawings are very similar DOM-based solutions. The chart with seats may have A LOT of elements, like some shapes for seats, text labels for rows, columns and sections, etc. So it may produce a lot of DOM elements. They can slow down the performance of the application.

2d canvas solution doesn’t need many DOM elements (it need just a canvas) so it can lead to a better performance comparing to SVG solution. Also if you are using Konva framework you already have some built-in functions that can help you bootstrap the application faster, like drag&drop, drag boundaries limits, caching to boost the performance (more on this later). react-konva npm package will allow us to make a React component.

Building the architecture

The chart structure will be loaded from the backend. The JSON has full information about available sections, seats, prices, etc.

The chart building starts from the data loading and making an initial layout for the chart:

const useFetch = (url) => {
  const [data, setData] = React.useState(null);
  React.useEffect(() => {
    fetch(url)
      .then((res) => res.json())
      .then((data) => setData(data));
  }, [url]);
  return data;
};

const MainStage = (props) => {
  const jsonData = useFetch('./seats-data.json');
  const width = 1000;
  const height = 1000;

  if (jsonData === null) {
    return <div>Loading...</div>;
  }

  return (
    <Stage width={width} height={height}>
      <Layer>
        {sections.map((item) => {
          return <Section section={item} />;
        })}
      </Layer>
    </Stage>
  );
};

Layout

The structure of the chart is simple. A concert hall consists of several sections (with different prices), each section has its own subsections and each subsection has many seats.

Now we need to decide how we will layout the chart and how we will set positions of all internal components.

There are two main approaches:

From top to bottom. When a parent container decides its own size and passes size to child components down to the tree.
From bottom to top. When a child component defines its own size manually and then the parent element adopts its own size and positions of children elements.

When you are building the web application with CSS you can use any of those methods or even combine them.

In general from bottom to top approach should work better, because the parent container doesn’t need to know full information about children elements and their sizes to define layout. But with Konva you can’t use that method directly because you have to set all positions of all elements manually. Let me show the issue:

<Stage width={width} height={height}>
  <Layer>
    {sections.map(item => {
      // we need to set position of each section
      // but if the height of each section is differrnt
      // and defined inside a Section component
      // how we will move set positions?
      return <Section y={??} section={item} />
    })}
  </Layer>
</Stage>

As I solution I decided to create a special file layout.js where I can define all the constants and the functions to calculate the size of any element:

export const SEAT_SIZE = 10;
export const SEATS_DISTANCE = 15;
export const SUBSECTION_PADDING = 30;

export const SECTION_TOP_PADDING = 40;

export const getSubsectionHeight = (subsection) => {
  const rows = Object.keys(subsection.seats_by_rows);
  return SEATS_DISTANCE * rows.length + SUBSECTION_PADDING * 2;
};

export const getSectionHeight = (section) => {
  return Math.max(...section.subsections.map(getSubsectionHeight)) + SECTION_TOP_PADDING;
};

That file will work as CSS definition for the whole chart layout. It is a single place where all constants are defined, so we don’t mess the rendering.

So sections drawing can look like this:

// where was the last section?
let lastSectionPosition = 0;

<Stage width={width} height={height}>
  <Layer>
    {sections.map((item) => {
      const height = layout.getSectionHeight(section);
      const position = lastSectionPosition + layout.SECTIONS_PADDING;
      lastSectionPosition = position + height;
      return <Section y={position} section={item} />;
    })}
  </Layer>
</Stage>;

Zooming

The chart can be large, if you are using it on the desktop, you can draw it as is, because you have plenty of space to make everything visible. But on mobile things become different.

We need to find a way to zoom out the whole chart to make it fully visible and zoom in to make it possible to interact with the particular seats.

Until the full chart is loading we don’t know what will be its size. We don’t know the number of seats, subsections, etc.

But, right after the chart is loaded and the rendering is finished, we can take and access to Konva nodes to calculate the size of the rendered chart and calculated required zoom:

const width = 1000;
const [scale, setScale] = React.useState(1);
const [scaleToFit, setScaleToFit] = React.useState(1);

React.useEffect(() => {
  if (!stageRef.current) {
    return;
  }
  const stage = stageRef.current;
  // client rect will return bounding box of the stage
  // it means the size of area, required to draw all the children
  const clientRect = stage.getClientRect();

  const scaleToFit = width / clientRect.width;
  setScale(scaleToFit);
  setScaleToFit(scaleToFit);
}, []);

return <Stage width={width} height={height} scaleX={scale} scaleY={scale}>;

As we know the scale required to fit everything into the screen, we can use it to easily toggle zoom on double tap or double click:

const toggleScale = React.useCallback(() => {
  if (scale === 1) {
    setScale(scaleToFit);
  } else {
    setScale(1);
  }
}, [scale, scaleToFit]);

Drawing the chart

The actual rendering of the chart is very straightforward. We are just using <Group> to organize sections, drawing background for each section as a rectangle and seats as circles.

As the result of initial drawing I have this:

Performance

After making the initial implementation of all basic functions I need to check how the app works in term of the performance.

One of the cool side of 2d canvas and Konva frameworks is the ability to cache any node (like a shape or a group) into an image. So the browser doesn’t need to redraw every node on an update. Because it can just draw a cached bitmap image. It is much faster for many cases.

In this app I used a caching for sections:

const Section = ({ x, y }) => {
  const containerRef = React.useRef();
  React.useEffect(() => {
    // recache group on any update
    containerRef.current.cache();
    // manually update Konva layer with the last updates
    containerRef.current.getLayer().batchDraw();
  });
  return (
    <Group y={y} x={x} ref={containerRef}>
      {/* internal components of the section */}
    </Group>
  );
};

When I tried to emulate low-level device on my MacBook the performance for drag&drop increased a lot from 5-10 fps to 50-60 fps!

Another way to improve the widget is to make it as small as possible in terms of dependencies. There is no sense to remove React from the solution because it is already used in the main client’s application. We can remove Konva dependency and draw everything with SVG, but it can drop realtime performance

The bad SVG performance is just a guess and may not be true. For large apps, it is better to make a prototype with all possible approaches and see which one works better.

The next step will is to make react-konva dependency as small as possible. But default Konva codebase is large, it has many shapes, filters, etc. In our case we don’t need all of these, accept some shapes. To make it smaller we can do this:

// instead of:
// import { Stage, Layer, Rect } from "react-konva";

// we will load minimal version of 'react-konva`
// minimal version has NO support for core shapes and filters
// if you want import a shape into Konva namespace you can just do this:
import 'konva/lib/shapes/Rect';
import 'konva/lib/shapes/Circle';
import 'konva/lib/shapes/Text';

// load minimal version of 'react-konva`
export { Stage, Layer, Group, Rect, Circle, Text } from 'react-konva/lib/ReactKonvaCore';

The result

At the end of the work, we have a fast and easy to use widget that works on mobile and desktop browsers.

You can try here (or open in a new tab):

Looking for help with a similar application? Contact me.

JavaScript loops - how to handle async/await

Tue, 31 Oct 2017 00:00:00 GMT

How to run async loops in sequence or in parallel?

Before doing asynchronous magic with loops I want to remind you how we write classical synchronous loops.

😐 Synchronous loop

Long time ago I was writing loops like this (probably you too):

for (var i=0; i < array.length; i++) {
  var item = array[i];
  // do something with item
}

It is good, it is fast, but it has many readability and maintenance issues. Then I used to use its better version:

array.forEach((item) => {
  // do something with item
});

JavaScript language is developing very fast. We have more features and new syntax. One of my favorite is async/await. I am using it more frequently now. And sometimes I have a situation where I need to do something with items in an array asynchronously.

🤗 Asynchronous loops

How to use await inside a loop? Let’s just write async function and await each task.

async function processArray(array) {
  array.forEach(item => {
    // define synchronous anonymous function
    // IT WILL THROW ERROR!
    await func(item);
  })
}

This code will throw a syntax error. Why? Because we can not use await inside synchronous function. As you can see “processArray” is async function. But anonymous function that we use for forEach is synchronous.

🤔 1. Don’t wait for result

How to fix previous issue? We can define anonymous function as asynchronous as well:

async function processArray(array) {
  array.forEach(async (item) => {
    await func(item);
  })
  console.log('Done!');
}

But forEach will not wait until all items are finished. It will just run tasks and go next. As a proof let’s write simple test:

function delay() {
  return new Promise(resolve => setTimeout(resolve, 300));
}

async function delayedLog(item) {
  // notice that we can await a function
  // that returns a promise
  await delay();
  console.log(item);
}
async function processArray(array) {
  array.forEach(async (item) => {
    await delayedLog(item);
  })
  console.log('Done!');
}

processArray([1, 2, 3]);

The output will be:

Done!
1
2
3

It can be ok if you don’t need to wait for the results. But in almost all cases this is not a good logic.

👊 2. Process array in sequence

To wait the result we should return back to old-school “for loop”, but this time we can use modern version with for..of construction (thanks to Iteration Protocol) for better readability:

async function processArray(array) {
  for (const item of array) {
    await delayedLog(item);
  }
  console.log('Done!');
}

This will give us expected output:

1
2
3
Done!

The code will handle each item one by one in series. But we can run it in parallel!

💪 3. Process array in parallel

We can slightly change the code to run async operations in parallel:

async function processArray(array) {
  // map array to promises
  const promises = array.map(delayedLog);
  // wait until all promises are resolved
  await Promise.all(promises);
  console.log('Done!');
}

This code will run many delayLog tasks in parallel. But be careful with very large array (too many tasks in parallel can be too heavy for CPU or memory).

Also don’t confuse “parallel” here with real threads and paralleling. That code will not guarantee real parallelism of execution. It depends on your item function (delayedLog in this demo). Network requests, webworkers and some other tasks can be executed in parallel.

That is all. Thanks for reading!

Optimizing React application with mobx

Fri, 20 Oct 2017 00:00:00 GMT

From 1.5 sec to 50–100ms (per update)

There is my short story about how I improved performance of the web-app by 10x with just a simple trick.

I was building web-based application for a client. It was captions (subtitles) review tool for videos. So you have a video, you have list of subtitles and you want to edit them.

One day I have loaded large projects into the app. The project had several hundreds of captions. The app was not fast, but it was working good enough. Until I click “delete” button of the first caption. I noticed that this operation was VERY slow. It took about 1.5 sec for deleting. This glitch was very noticeable. Why was it slow? After some investigation, I found that React has to rerender ALL captions on delete. You may ask why, if it is just deleting? The thing is — each caption should show its index (see in top left). So when you delete the first caption, indexes of all other captions need to be updated.

As a proof I deleted the last element, it was very fast. It may still look strange why updating of indexes is slow. Even if react didn’t need to touch a lot of elements (pfuuu, just several hundreds), it needs to do a LOT of work under the hood. It needs to call render for each element, it needs to create a new tree of virtual DOM and it needs to calculate changes and it needs to apply them. This operation tooks long time because one caption component has many elements inside.

I started to think to use react-virtualized for hiding invisible captions. This should 100% solve the issue. But, the app has some features about scrolling list in several conditions. To calculate correct scrolling position I needed all elements to be in the DOM (otherwise calculation is too complex). As I have some legacy code, this change may take too long time to implement and test and it may break the app in some cases.

So I continued to think about a faster solution. In that project, I was using mobx for state management. The cool thing about it is observable values and ability to create React components that OBSERVE changes. So I thought: I need to update item element but how to update index directly without calling render of the whole caption?

This is how part for render function was looking before changes:

mobx-react automatically track usage of properties in render function. “index” property changed — component updated.

But here is quick and simple fix. Let us create special component for displaying JUST index:

And then we will use this component inside render of whole item:

What did it change? In this case I am not using “index” property directly inside item render. So when I updated the index this component is not updated (it doesn’t observe that changes). BUT child component will be updated (directly and very fast), because it listens to changes of “index” property.

Notice that this approach will not work, if you will use index directly here, like <IndexTest index={segment.index}/>

Performance improved A LOT with this simple refactoring.

Every time you want to fix performance of a web-app remember to be smart and measure the performance. In my case one item element has many properties (index, text, start time, end time and many other). And I didn’t move rendering of all of these properties into separate components. It just make no sense.

Question: what is the best way to apply something similar if you are using pure setState or redux architecture? I didn’t have anything beautiful in mind, do you?

Looking of help with react performance? Send me a message.

Hit Region Detection For HTML5 Canvas And How To Listen To Click Events On Canvas Shapes

Wed, 22 Feb 2017 00:00:00 GMT

Do you need a simple onClick for your canvas shapes? But canvas doesn’t have API for such listeners. You can listen to events only on whole canvas, not on part of it. I will describe two main approaches how to go around this problem.

NOTE! I will not use addHitRegion API because for the current moment (year 2017) it is still unstable and not fully supported. But you may take a look at it.

Let start from simple html5 canvas graphics. Imagine we want to draw several circles on a page.

const canvas = document.getElementById('canvas');
const ctx = canvas.getContext('2d');

// create circles to draw
const circles = [
  {
    x: 40,
    y: 40,
    radius: 10,
    color: 'rgb(255,0,0)'
  },
  {
    x: 70,
    y: 70,
    radius: 10,
    color: 'rgb(0,255,0)'
  }
];

// draw circles
circles.forEach(circle => {
  ctx.beginPath();
  ctx.arc(circle.x, circle.y, circle.radius, 0, 2 * Math.PI, false);
  ctx.fillStyle = circle.color;
  ctx.fill();
});

Open demo

Now we can simply listen to clicks on a WHOLE canvas:

canvas.addEventListener('click', () => {
   console.log('canvas click');
});

But we want to listen to clicks on a circle. How to do this? How to detect that we clicked on a circle?

Approach #1 — use power of math

As we have information about our circle’s coordinates and sizes we can simply use mathematics to detect a click on a circle with trivial calculations. All we need is to get mouse position from click event and check all circles for intersection:

function isIntersect(point, circle) {
  return Math.sqrt((point.x-circle.x) ** 2 + (point.y - circle.y) ** 2) < circle.radius;
}

canvas.addEventListener('click', (e) => {
  const pos = {
    x: e.clientX,
    y: e.clientY
  };
  circles.forEach(circle => {
    if (isIntersect(mousePoint, circle)) {
      alert('click on circle: ' + circle.id);
    }
  });
});

This approach is very common and widely used in many projects. You can easily find math functions for more complex shapes such as rectangles, ellipses, polygons, etc.

This approach is very good and it can be ultra fast if you don’t have a huge number of shapes.

But it is very hard to use this approach for very complex shapes. For example, you are using lines with quadratic curves.

Approach #2 — emulating hit region

The idea of hit regions is simple — we just need to get pixel under clicked area and find a shape that has the same color:

function hasSameColor(color, circle) {
  return circle.color === color;
}

canvas.addEventListener('click', (e) => {
  const mousePos = {
    x: e.clientX - canvas.offsetLeft,
    y: e.clientY - canvas.offsetTop
  };
  // get pixel under cursor
  const pixel = ctx.getImageData(mousePos.x, mousePos.y, 1, 1).data;

  // create rgb color for that pixel
  const color = `rgb(${pixel[0]},${pixel[1]},${pixel[2]})`;

  // find a circle with the same colour
  circles.forEach(circle => {
    if (hasSameColor(color, circle)) {
      alert('click on circle: ' + circle.id);
    }
  });
 });

But exactly this approach will not work, because it may have shapes with the same color, right? To avoid such collision we should create a special “hit graph” canvas. It will have almost the same shapes but each shape will have a unique color. So we need to generate random colors for each circle:

// colorsHash for saving references of all created circles
const colorsHash = {};

function getRandomColor() {
const r = Math.round(Math.random() * 255);
const g = Math.round(Math.random() * 255);
const b = Math.round(Math.random() * 255);
return `rgb(${r},${g},${b})`;
}



const circles = [{
 id: '1', x: 40, y: 40, radius: 10, color: 'rgb(255,0,0)'
}, {
 id: '2', x: 100, y: 70, radius: 10, color: 'rgb(0,255,0)'
}];

// generate unique colors
circles.forEach(circle => {
 // repeat until we find trully unique colour
 while(true) {
    const colorKey = getRandomColor();
    // if colours is unique
    if (!colorsHash[colorKey]) {
       // set color for hit canvas
       circle.colorKey = colorKey;
       // save reference 
       colorsHash[colorKey] = circle;
       return;
    }
 }
});

After that, we need to draw each shape TWICE. First on visible canvas, then on “hit” canvas.

circles.forEach(circle => {
  // draw on "scene" canvas first
  ctx.beginPath();
  ctx.arc(circle.x, circle.y, circle.radius, 0, 2 * Math.PI, false);
  ctx.fillStyle = circle.color;
  ctx.fill();
  
  // then draw on offscren "hit" canvas
  hitCtx.beginPath();
  hitCtx.arc(circle.x, circle.y, circle.radius, 0, 2 * Math.PI, false);
  hitCtx.fillStyle = circle.colorKey;
  hitCtx.fill();
});

Now when you click on canvas all you need is to take a pixel on the hit canvas and find a shape with the same color. And this actions are ultra fast and you don’t need to iterate over ALL shapes. Also, it doesn’t matter how complex your shape is. Draw whatever you want and just use different colors for each shape.

Demo with the full code

Which approach is better?

It depends. The main bottleneck of the second “hit” approach is that you have to draw shapes twice. So performance can drop twice! But drawing on hit canvas can be simpler. You can skip shadows and strokes there, you may simplify some shapes, for instance, replace a text with just a rectangle. But AFTER drawing is finished this approach can be ultra fast. Because taking a pixel and accessing to a hash of stored shapes is very fast operation.

Can they be used together?

Sure. Several canvas libs use such mixed approach.

It works in this way:

For each shape, you have to calculate simplified bounding rectangles (x, y, width, height). Then you use first “math” approach for filtering shapes that have an intersection with mouse position and bounding rectangle. After that you can draw hit and test intersection with the second approach for more accurate result.

Why not just use SVG for such case?

Because sometimes canvas can be more performant and more appropriate for your high-level task. Again, it depends on a task. So canvas vs SVG is out of the context of this post. And if you want to use canvas and have a hit detection you have to use something, right?

What about other events? Like mousemove, mouseenter, etc?

You just need to add some extra code into described approaches. And you can emulate all other events once you can 100% detect a shape under the mouse.

Are there any good ready-to-use solutions?

Sure. Just try to google “html5 canvas framework”. But my personal recommendation is http://konvajs.github.io/. Almost forgot, I am a maintainer of this library. Konva uses only second approach and it has support for all mouse and touch events that we usually have for DOM elements (and even more, like drag&drop).

Optimizing react-redux store for high performance updates

Thu, 22 Dec 2016 00:00:00 GMT

Looking of help with react performance? Send me a message.

Optimizing react-redux store for high performance updates

How to structure your redux store to improve rendering performance?

This post is continuation of Tips to optimise rendering of a set of elements in React

Note: this post designed specially for React + Redux applications. Also, this tip works for react-redux v5. I was not able to apply the same technique with react-redux v4. Probably some of its inner listeners were too slow.

The very common way to store some set of elements in an application is to store items as an array:

const state = {
  targets: [{id: 'target1', radius: 10}, {id: 'target2', radius: 2}]
};

Probably you will have a component in your application that will render the set of elements:

const TargetsList = connect(state => ({targets: state.targets}))(
  ({ targets }) =>
    <ul>
      { targets.map((target) => <TargetView key={target.id} target={target} />) }
    </ul>
);

If you need to update an item in the list you will have to update whole list:

function appReducer(state, action) {
  if (action.type === 'UPDATE') {
    return {
      target: state.target.map((target) => {
        if (target.id === action.id) {
           return {
              ...target,
              radius: action.radius
           };
        } else {
          return target;
        }
      }) 
    }
  }
  // some other code
}

Every update of an item will produce update of the whole “TargetsList” view.

I created a simple demo with such approach (with testing code): http://codepen.io/lavrton/pen/xRgYbL

Updating an item in the array of size 1000 takes about ~21ms on my machine. In the previous post, I described an approach of optimizing such rendering by making child components “smarter” and adding some logic into “shouldComponentUpdate” of list component. But almost the same result can be achieved much simpler by changing state shape.

How to optimize?

If you are using https://github.com/reactjs/react-redux you can simply improve performance by changing state shape to:

const state = {
  targetsOrder: ['id-1', 'id-2'],
  targets: {
    'id-1': { id: 'id-1', radius: 10 },
    'id-2': { id: 'id-2', radius: 20 },
  }
};

Then we will need to change “TargetsList” a bit:

const TargetsList = connect(state => ({targetsOrder: state.targetsOrder}))(
  ({ targetsOrder }) =>
    <ul>
      { targetsOrder.map((id) => <TargetView key={id} targetId={id} />) }
    </ul>
);

Please, note, that in this case, I am passing item ID into child view, not the whole item. In this case “TargetView” can not be “dumb” component. It should be connected to store:

onst TargetView = connect(
  (state, ownProps) => ({target: state.targets[ownProps.targetId]})
)(({ target }) => {
  // your render logic
  return <Circle radius={target.radius} />;
});

As TargetView is connected to store, it will update itself when its “target” data is updated. It is important that if we update an item in a list, “TargetList” will be NOT updated, because “targetsOrder” is the same. In the same cases, it may improve performance a lot. Demo: http://codepen.io/lavrton/pen/ZBLrWp

Updating an item takes 2.2ms on my machine. It is almost 10x faster then with previous shape of the state!

Looking of help with react performance? Send me a message.

Progressive loading for modern web applications via code splitting

Wed, 20 Jul 2016 00:00:00 GMT

Progressive loading for modern web applications via code splitting

Are your users tired of waiting when your app is loading and they close the tab? Let’s fix it with the progressive loading!

I will use webpack for bundling and React for a demonstration.

I am compiling and bundling all my javascript files (sometimes css and images too) into ONE HUGE bundle.js. I guess you are doing this too, aren’t you? It is a pretty common approach for making modern web applications.

But this approach has one (sometimes very important) drawback: first loading of your app may take too much time. As a web browser have to (1) load large file and (2) parse a lot of javascript code. And loading can take really much time if a user has bad internet connection. Also, your bundled file can have components that user will never see (e.g. user will never open some parts of your application).

Progressive Web Apps?

One of the good solutions for better UX is Progressive Web App. Google this term if you don’t know it yet. There are tons of good posts and videos about it. So Progressive Web has several core ideas, but right now I want to focus on Progressive Loading and implement it .

The idea of Progressive Loading is very simple:

Make “initial load” as fast as possible.
Load UI components only when they are required.

Let us assume we have React Application that draws some charts on a page:

import React from 'react';

import LineChart from './LineChart';
import BarChart from './BarChart';

export default class App extends React.Component {
    // do not show charts on first loading
    state = {
        showCharts: false
    };

    // toggle charts
    handleChange = () => {
        this.setState({
            showCharts: !this.state.showCharts
        });
    }

    render() {
        return (
            <div>
                Show charts:
                <input
                  type="checkbox"
                  value={this.state.showCharts}
                  onChange={this.handleChange}
                />

                {
                    this.state.showCharts ?
                    <div><LineChart/><BarChart/></div>
                    : null
                }
            </div>
        );
    }
}

Chart components are very simple:

// LineChart.js
import React from 'react';
import {Stage, Layer, Line} from 'react-konva';

export default () => (
    <Stage width={100} height={100}>
        <Layer>
            <Line stroke="green" points={[0, 0, 20, 90, 50, 20, 100, 100]}/>
        </Layer>
    </Stage>
);

// BarChart.js
import React from 'react';
import {Stage, Layer, Rect} from 'react-konva';

export default () => (
    <Stage width={100} height={100}>
        <Layer>
            <Rect fill="red" width={20} height={20}/>
            <Rect fill="blue" x={50} width={20} height={60}/>
        </Layer>
    </Stage>
);

These charts can be very heavy. Both of them have react-konva as a dependency (and konva framework as a dependency of react-konva).

Please note that LineChart and BarChart are not visible on the first load. To see them a user needs to toggle checkbox:

So it is possible that the user will NEVER toggle that checkbox. And this is a very common situation in real world web application: when a user never opens some parts of the app (or open them later). But with a current approach, we have to bundle all components and all their dependencies into one file. In this example we have: root App component, React, Chart components, react-konva, konva.

280kb for bundle.js and 3.5 seconds for initial loading with a regular 3g connection.

Implementing Progressive Loading

How can we remove these chart components from bundle.js and load them later and draw something meaningful as fast as possible? Say hello to good old AMD (asynchronous module definition)! And webpack has good support for code splitting.

I suggest to define HOC (higher order component) that will load chart only when a component is mounted into DOM (with componentDidMount lifecycle callback). Let’s define LineChartAsync.js:

import React from 'react';

export default class AsyncComponent extends React.Component {
    state = {
        component: null
    }
    componentDidMount() {
        // load component on mount
        require.ensure([], (require) => {
            // !IMPORTANT! we can not use here something like:
            //    require(this.props.path).default;
            // because webpack cannot  statically analyse such imports
            // so we should explicitly import required module
            const Component = require('./LineChart').default;
            this.setState({
                component: Component
            });
        });
    }
    render() {
        if (this.state.component) {
            return <this.state.component/>
        }
        return (<div>Loading</div>);
    }
}

Then instead of

import LineChart from ‘./LineChart’;

We should write:

import LineChart from ‘./LineChartAsync’;

Let us see what we have after bundling:

We have bundle.js that includes a root App component and React.

1.bundle.js and 2.bundle.js are generated by webpack and they include LineChart and BarChart. But, wait, why is the total sum bigger? 143kb+143kb+147kb = 433kb vs 280kb from previous approach. That is because dependencies of LineChart and BarChart are included TWICE (react-konva and konva defined in both 1.bundle.js and 2.bundle.js), we can avoid this with webpack.optimize.CommonsChunkPlugin:

new webpack.optimize.CommonsChunkPlugin({
    children: true,
    // (use all children of the chunk)
    async: true,
    // (create an async commons chunk)
}),

Now dependencies of LineChart and BarChart are moved in another file 3.bundle.js, total size is almost the same (289kb). Now we have 1.75 seconds for initial loading. It is much better then 3.5 seconds.

Refactoring

To make the code better I would like to refactor LineChartAsync and BarChartAsync. First, let’s define basic AsyncComponent:

import React from 'react';

export default class AsyncComponent extends React.Component {
    state = {
        component: null
    }
    componentDidMount() {
        this.props.loader((componentModule) => {
          this.setState({
              component: componentModule.default
          });
        });
    }
    renderPlaceholder() {
      return <div>Loading</div>;
    }
    render() {
        if (this.state.component) {
            return <this.state.component/>
        }
        return (this.props.renderPlaceholder || this.renderPlaceholder)();
    }
}

AsyncComponent.propTypes = {
    loader: React.PropTypes.func.isRequired,
    renderPlaceholder: React.PropTypes.func
};

And BarChartAsync (and LineChartAsync) can be rewritten into simpler component:

import React from 'react';
import AsyncComponent from './AsyncComponent';

const loader = (cb) => {
  require.ensure([], (require) => {
      cb(require('./BarChart'))
  });
}

export default (props) =>
  <AsyncComponent {...props} loader={loader}/>

But we can improve Progressive Loading even more! When application is initially loaded we can schedule loading of additional component on background, so it is possible that they will be loaded before user toggled checkbox

import React from 'react';
import AsyncComponent from './AsyncComponent';
import sceduleLoad from './loader';

const loader = (cb) => {
  require.ensure([], (require) => {
      cb(require('./BarChart'))
  });
}

sceduleLoad(loader);

export default (props) =>
  <AsyncComponent {...props} loader={loader}/>

And loader.js will be something like this:

const queue = [];
const delay = 300;

let isWaiting = false;


function requestLoad() {
    if (isWaiting) {
      return;
    }
    if (!queue.length) {
      return;
    }
    const loader = queue.pop();
    isWaiting = true;
    loader(() => {
      setTimeout(() => {
        isWaiting = false;
        requestLoad();
      }, delay)
    });
}

export default function sceduleLoad(loader) {
  queue.push(loader);
  requestLoad();
}

Also, we can define components that will visible on the first screen, but in fact loaded asynchronously later and a user may see beautiful placeholder while a component is loading. Please note that placeholder is not for API call. It is exactly for loading component’s module (its definition and all its dependencies).

const renderPlaceholder = () =>
    <div style={{textAlign: ‘center’}}>
        <CircularProgress/>
    </div>
export default (props) =>
    <AsyncComponent
       {…props}
       loader={loader}
       renderPlaceholder={renderPlaceholder}
    />

Conclusion

As a result of all improvements:

Initial bundle.js has a smaller size. That means a user will see some working UI components faster.
Additional components can be loaded asynchronously in the background.
While a component is loading it can be replaced with some placeholder components .
For exactly this approach Webpack is required. But you can use it not only with React, but with other frameworks too.

Take a look https://github.com/lavrton/Progressive-Web-App-Loading for full source and webpack configurations.

Tips to optimise rendering of a set of elements in React

Wed, 11 May 2016 00:00:00 GMT

Looking of help with react performance? Send me a message.

Tips to optimise rendering of a set of elements in React

There is a guide to increase React performance. The advanced tip will increase speed by 20 times.

Rendering a list of some elements in a page is a common task for almost any web-app. In this post I would like to show how to improve performance for that case.

For a test example we will create app that draws a set of targets (circles) on the <canvas> element. I will use Redux for a data storage. But these tips can be applied for many other state managing approaches. Also, you can use these tips with react-redux, but for a clear explanation I will not use it.

Let me start from store definition.

function generateTargets() {
    return _.times(1000, (i) => {
        return {
            id: i,
            x: Math.random() * window.innerWidth,
            y: Math.random() * window.innerHeight,
            radius: 2 + Math.random() * 5,
            color: Konva.Util.getRandomColor()
        };
    });
}

// for test case our logic will be very simple
// just one action UPDATE for updating radius of a target
function appReducer(state, action) {
   if (action.type === 'UPDATE') {
       const i = _.findIndex(state.targets, (t) => t.id === action.id);
       const updatedTarget = {
           ...state.targets[i],
           radius: action.radius
       };
       state = {
           targets: [
               ...state.targets.slice(0, i),
               updatedTarget,
               ...state.targets.slice(i + 1)
           ]
       }
   }
   return state;
}

const initialState = {
    targets: generateTargets()
};
// create redux store
const store = Redux.createStore(appReducer, initialState);

Then define our application rendering. I will use react-konva for canvas rendering.

function Target(props) {
    const {x, y, color, radius} = props.target;
    return (
        <Group x={x} y={y}>
            <Circle
                radius={radius}
                fill={color}
            />
            <Circle
                radius={radius * 1 / 2}
                fill="black"
            />
            <Circle
                radius={radius * 1 / 4}
                fill="white"
            />
        </Group>
    );
}

// top component with list of targets
class App extends React.Component {
    constructor(...args) {
        super(...args);
        this.state = store.getState();
        // subscibe to all state updates
        store.subscribe(() => {
            this.setState(store.getState());
        });
    }
    render() {
        const targets = this.state.targets.map((target) => {
            return <Target key={target.id} target={target}/>;
        });
        const width = window.innerWidth;
        const height = window.innerHeight;
        return (
            <Stage width={width} height={height}>
                <Layer hitGraphEnabled={false}>
                    {targets}
                </Layer>
            </Stage>
        );
    }
}

Open demo

Now let’s create a simple test script that will run several updates on one target:

const N_OF_RUNS = 500;
const start = performance.now();
_.times(N_OF_RUNS, () => {
    const id = 1;
    let oldRadius = store.getState().targets[id].radius;
    // update redux store
    store.dispatch({type: 'UPDATE', id, radius: oldRadius + 0.5});
});
const end = performance.now();

console.log('sum time', end - start);
console.log('average time', (end - start) / N_OF_RUNS);

Let’s run test script for the app without any optimisations. On my machine, an update will take ~21ms.

This time doesn’t include canvas drawing, only redux + react code because react-konva is drawing objects only on the next animation frame tick. We are not interested in a canvas optimisation right now, it can be a subject of another post.

So ~21ms for 1000 elements is pretty good performance. If we update elements rarely we can keep this code the way it is.

But I had a case where I need to execute update very frequently (on every mouse move during drag&drop). For 60 FPS animation, each update should take no more than 16ms. So 21ms is not so good for this case (plus will have canvas drawing).

How can we optimise rendering?

Don’t update unchanged elements

This is a first rule to improve React performance. All we need is to implement shouldComponentUpdate for Target Element:

class Target extends React.Component {
    shouldComponentUpdate(newProps) {
        return this.props.target !== newProps.target;
    }
    render() {
        const {x, y, color, radius} = this.props.target;
        return (
            <Group x={x} y={y}>
                <Circle
                    radius={radius}
                    fill={color}
                />
                <Circle
                    radius={radius * 1 / 2}
                    fill="black"
                />
                <Circle
                    radius={radius * 1 / 4}
                    fill="white"
                />
            </Group>
        );
    }
}

Open demo

Now it takes ~4ms vs 21ms. This is much better. But can we do more? In my real app even after this optimisation the performance was still bad.

Advanced tuning

Now take a look into render() function of App component. The thing I don’t like in this code is that render() of app component will be called on EACH update.

That means we have more that 1000 calls for React.createElement for each target. In this case, it works fast but in larger apps it can be slow.

Why should we rerender the whole list if we know that only particular components are updated? Can we just directly update them?

Make child components smarter

The idea is simple:

Don’t update the app component if a list has the same number of elements and order of elements is the same
Children components should update itself if data is changed

So “Target” component should subscribe to store and track changes:

class Target extends React.Component {
    constructor(...args) {
        super(...args);
        this.state = {
            target: store.getState().targets[this.props.index]
        };
        // subscibe to all state updates
        this.unsubscribe = store.subscribe(() => {
            const newTarget = store.getState().targets[this.props.index];
            if (newTarget !== this.state.target) {
                this.setState({
                    target: newTarget
                });
            }
        });
    }
    shouldComponentUpdate(newProps, newState) {
         return this.state.target !== newState.target;
    }
    componentWillUnmount() {
      this.unsubscribe();
    }
    render() {
        const {x, y, color, radius} = this.state.target;
        return (
            <Group x={x} y={y}>
                <Circle
                    radius={radius}
                    fill={color}
                />
                <Circle
                    radius={radius * 1 / 2}
                    fill="black"
                />
                <Circle
                    radius={radius * 1 / 4}
                    fill="white"
                />
            </Group>
        );
    }
}

And implement shouldComponentUpdate for App component:

Result after that changes - http://codepen.io/lavrton/pen/bpxZjy

0.25ms for an update is much better now.

Bonus tip

Use https://github.com/mobxjs/mobx to skip all of this subscribing code. Same app using mobx (http://codepen.io/lavrton/pen/WwPaeV) - 0.16ms. About 1.5x faster than the previous result (and difference will be much more if you have more elements). And code is much simpler:

const {Stage, Layer, Circle, Group} = ReactKonva;
const {observable, computed} = mobx;
const {observer} = mobxReact;

class TargetModel {
    id = Math.random();
    @observable x = 0;
    @observable y = 0;
    @observable radius = 0;
    @observable color = null;
    constructor(attrs) {
        _.assign(this, attrs);
    }
}

class State {
    @observable targets = [];
}


function generateTargets() {
     _.times(1000, (i) => {
        state.targets.push(new TargetModel({
            id: i,
            x: Math.random() * window.innerWidth,
            y: Math.random() * window.innerHeight,
            radius: 2 + Math.random() * 5,
            color: Konva.Util.getRandomColor()
        }));
    });
}

const state = new State();
generateTargets();


@observer
class Target extends React.Component {
    render() {
        const {x, y, color, radius} = this.props.target;
        return (
            <Group x={x} y={y}>
                <Circle
                    radius={radius}
                    fill={color}
                />
                <Circle
                    radius={radius * 1 / 2}
                    fill="black"
                />
                <Circle
                    radius={radius * 1 / 4}
                    fill="white"
                />
            </Group>
        );
    }
}

@observer
class App extends React.Component {
    render() {
        const targets = state.targets.map((target) => {
            return <Target key={target.id} target={target}/>;
        });
        const width = window.innerWidth;
        const height = window.innerHeight;
        return (
            <Stage width={width} height={height}>
                <Layer hitGraphEnabled={false}>
                    {targets}
                </Layer>
            </Stage>
        );
    }
}

ReactDOM.render(
  <App/>,
  document.getElementById('container')
);


// update one target
state.targets[1].radius += 0.5

Looking of help with react performance? Send me a message.

Using React With Canvas Element

Mon, 01 Feb 2016 00:00:00 GMT

How to use React with canvas element?

We have React that helps us to work with massive mutable DOM in a nice immutable functional style. That is awesome.

But I would like to describe a way of using React that helps you to work with mutable object representation of “immutable” <canvas> element. Yes, it sounds strange. But it works really well.

Motivation

I am using <canvas> element a lot. I made several complex applications which use <canvas> as a main view component. Vanilla canvas without any frameworks/libraries can be really difficult for complex applications. So I started to use canvas frameworks a lot. Now I maintain Konva 2d canvas Framework.

Konva helps a lot, but still now not so good, as I wanted. Also I started to use React in my applications and I really like to use it. So I was thinking how can I use React for drawing graphics on canvas.

Native canvas inside React

It is very easy to access <canvas> context in React component and draw any graphics:

class CanvasComponent extends React.Component {
    componentDidMount() {
        this.updateCanvas();
    }
    updateCanvas() {
        const ctx = this.refs.canvas.getContext('2d');
        ctx.fillRect(0,0, 100, 100);
    }
    render() {
        return (
            <canvas ref="canvas" width={300} height={300}/>
        );
    }
}
ReactDOM.render(<CanvasComponent/>, document.getElementById('container'));

Demo: http://jsbin.com/xituko/edit?js,output

It works nice for trivial examples. But for complex applications this approach is not so good because we can not use full power of reusable React components.

// “reusable component”
function rect(props) {
    const {ctx, x, y, width, height} = props;
    ctx.fillRect(x, y, width, height);
}
class CanvasComponent extends React.Component {
    componentDidMount() {
        this.updateCanvas();
    }
    componentDidUpdate() {
        this.updateCanvas();
    }
    updateCanvas() {
        const ctx = this.refs.canvas.getContext('2d');
        ctx.clearRect(0,0, 300, 300);
        // draw children “components”
        rect({ctx, x: 10, y: 10, width: 50, height: 50});
        rect({ctx, x: 110, y: 110, width: 50, height: 50});
    }
    render() {
         return (
             <canvas ref="canvas" width={300} height={300}/>
         );
    }
}
ReactDOM.render(<CanvasComponent/>, document.getElementById('container'));

And what about React’s lifecycle methods (like shouldComponentUpdate etc) and “all representation inside render function”?

Implementation

I really like React approach for building apps. So I made react binding to Konva objects. And now I can draw canvas stage with different graphic shapes and use it in my React application.

npm install react konva react-konva --save

Then

import React from 'react';
import ReactDOM from 'react-dom';
import {Layer, Rect, Stage, Group} from ‘react-konva’;
 
class MyRect extends React.Component {
    constructor(...args) {
      super(...args);
      this.state = {
        color: 'green'
      };
      this.handleClick = this.handleClick.bind(this);
    }
    handleClick() {
      this.setState({
        color: Konva.Util.getRandomColor()
      });
    }
    render() {
        return (
            <Rect
                x={10} y={10} width={50} height={50}
                fill={this.state.color}
                shadowBlur={10}
                onClick={this.handleClick}
            />
        );
    }
}
function App() {
    return (
      <Stage width={700} height={700}>
        <Layer>
            <MyRect/>
        </Layer>
      </Stage>
    );
}
ReactDOM.render(<App/>, document.getElementById('container'));

Demo: http://jsbin.com/camene/8/edit?html,js,output

Thanks https://github.com/reactjs/react-art for providing good examples of custom React components.

Comparisons

react-konva vs react-canvas

react-canvas is a completely different react plugin. It allows you to draw DOM-like objects (images, texts) on canvas element in very performant way. It is NOT about drawing graphics, but react-konva is exactly for drawing complex graphics on <canvas>element from React.

react-konva vs react-art

react-art allows you to draw graphics on a page. It also supports SVG for output. But it has no support of events of shapes.

react-konva vs vanilla canvas

Performance is one of the main buzz word in react hype.

I made this plugin not for performance reasons. Using vanilla <canvas> should be more performant because while using react-konva you have Konva framework on top of <canvas> and React on top of Konva. But I made this plugin to fight with application complexity. Konva helps here a lot (especially when you need events for objects on canvas, like “click” on shape, it is really hard to do with vanilla canvas). But React helps here much more as it provides very good structure for your codebase and data flow.

Features

react-konva works on top of Konva so it has support for all core shapes: Circle, Rect, Ellipse, Line, Sprite, Image, Text, TextPath, Star, Ring, Arc, Label, SVG Path, RegularPolygon, Arrow and you can create your own custom shapes. Also you can use build in drag&drop support, tweening, animations, filters, caching system, desktop and mobile events (mouseenter, click, dblclick, dragstart, dragmove, dragend, tap, dbltap, etc) and much more.

// custom shape example
function MyShape() {
  return (
     <Shape fill=”#00D2FF” draggable
         sceneFunc={function (ctx) {
             ctx.beginPath();
             ctx.moveTo(20, 50);
             ctx.lineTo(220, 80);
             ctx.quadraticCurveTo(150, 100, 260, 170);
             ctx.closePath();
             // Konva specific method
             ctx.fillStrokeShape(this);
         }}
     />
  );
}

Demo: http://jsbin.com/gakadi/4/edit?html,js,output

// events example
class MyCircle extends React.Component {
    constructor(…args) {
        super(…args);
        this.state = { isMouseInside: false};
        this.handleMouseEnter = this.handleMouseEnter.bind(this);
        this.handleMouseLeave = this.handleMouseLeave.bind(this);
    }
    handleMouseEnter() {
        this.setState({ isMouseInside: true});
    }
    handleMouseLeave() {
        this.setState({ isMouseInside: false});
    }
    render() {
        return (
            <Circle
                x={100} y={60} radius={50}
                fill=”yellow” stroke=”black”
                strokeWidth={this.state.isMouseInside ? 5 : 1}
                onMouseEnter={this.handleMouseEnter}
                onMouseLeave={this.handleMouseLeave}
            />
        );
    }
}

Demo: http://jsbin.com/tekopu/3/edit?js,output

Links:

Github repo: https://github.com/lavrton/react-konva

Konva framework: http://konvajs.github.io/