Demo Videos

After all these years, I decided to add some demo videos.

• Virtual Wigwam

 

• Rotating horse

• Bringing user-created 2D image to 3D space

• Adding texture

 

Week of 7/30/2014-8/5/2014

It has been the final week as a summer research assistant!

• Summary of the project 

I wrote a summary document of the project and it is available here.
Also, an interactive demo is available here.

• Knitting application 

 A knitting application is implemented as follows. This application uses a lot of small 3D objects, so it is a  good performance benchmark. It draws 11 * 21 * 2 = 462 cones in about 3 seconds on my laptop (ThinkPad T430s).

This application consumes about 30% of CPU even after the rendering and makes the user interface somewhat slow. To speed up the rendering speed, we may need to use WebGLRenderer instead of CanvasRenderer, which we are currently using.

Week of 7/23/2014 - 7/29/2014

• Creating common 3D shapes using 3D pen

 To let users stamp common 3D shapes, the following codelets (blocks) are added in the "Pen" palette.

Here are some example shapes created from the above codelets:

A "knot" codelet is intended to be used for a 3D version of the knitting application (http://community.csdt.rpi.edu/projects/38/). For the detailed parameter usage of the knit codelet, please refer to this interactive example (http://www.smartjava.org/ltjs/chapter-05/08-basic-3d-geometries-torus-knot.html).

Currently, stampled shapes take over the following properties from a 3D sprite (in above example, a bird): color, position, rotation, scale, and texture. Thus, if you want to change those properties for shapes created from 3D pen, first you can apply 3D operations to the 3D sprite and then stamp any shapes you want.

Using the arc codelet, we can develop a VirtualWigwam application as follows:

Hereby we achieved the goal we set in the beginning of the summer!

• 3D shape display control for speed up the rendering

In the current design, the rendering process gets increasingly slow, when you add more 3D shapes. For example, when we want to display N arcs created by 3D pen, rendering steps go like the following:

- 1st step: rendering 1st arc
- 2nd step: rendering 1st and 2nd arcs
- 3rd step: rendering 1st, 2nd, and 3rd arcs
   ....
- Nth step: rendering 1st, 2nd, ..., and Nth arcs

Thus, it costs you N*(N+1)/2 times of rendering the arc. This is unavoidable because if we change anything in the 3D space and make it visible, we need to request Three.js to update the entire canvas for 3D. Three.js might be doing something intelligent to avoid rendering the same objects over and over again, but at least the above sequence is what we request from the application's perspective.

Having said that, if we do not need to display intermediate results from the 1st - N-1th renderings, we can just skip these steps and display the Nth result only. Based on that idea, I have implemented a "hide pen" and a "show pen" codelets, which stops and resumes rendering 3D shapes by 3D pen.
Using these codelets just as the following, the system just renders arcs N times, not N*(N+1)/2 times.
 
  hide pen
  repeat N
    arc width: 100 height: 100
  show pen
 

• Switching codelets depending on if the sprite is 2D or 3D

 There are codelets that are applicable only for 2D, or 3D. So depending on the status of the sprite (i.e., 2D or 3D), showing codelets that work for the selected sprite makes sense. For example, when a 2D sprite is selected, the "pen" palette shows the following codelets:

 

On the other hand, when a 3D sprite is selected the same palette looks as follows:

 

• Concluding remarks

My summer job ends on August 1st. For the rest of the week, I will work on a document that describes the implementation of the 3D system for C-Snap.

I have been really enjoying this project and am reasonably happy for the achievements I made including the VirtualWigwam.

Hope someone takes over my work and further enhances this project!

Week of 7/16/2014 - 7/22/2014

• Texture mapping

Texture mapping is to apply an image to all the surface of a 3D object. Any regular images such as JPEG, PNG, GIF, ..., etc. can be textures. After loading a 3D costume of your choice, a texture can be selected from the newly added "Textures..." menu as follows:

  

Here are some examples of 3D objects with textures:

• Switch from 2D to 3D

After implemented the texture mapping function, I realized it was easy to turn 2D objects into 3D  by applying a texture to a 3D plane. So, I added a new switch to bring 2D objects into the 3D space  as follows:

Here are some examples of 2D objects in the 3D space:

 A dog is created from a 2D costume whereas a smiley is created from a user-painted image.

• To Do

Here is the updated To Do list:

    1. Drag & drop
    2. Nesting
    3. Thumbnail display
    4.  ✔ Overlapping with 2D object
    5.  ✔ Transformation (move x y z)
    6.  ✔ Rotation (rotate n degrees in x, m in y, l in z)
    7.  ✔ Scaling
    8.  ✔ Textures (apply this image to all the surface)
    9. 3D pen (render common 3D shapes such as cube, arc, sphere, cone, ..., and so on)
By implementing the switch from 2D to 3D, 4 is actually done. Next week, I'll be working on 9 and that would conclude my summer job.

Weeks of 7/9/2014 - 7/15/2014

I have been trying to bring "real 3D" to C-Snap since last week, and now it works great.

• New implementation 

In the new implementation, one layer is dedicated exclusively for 3D objects as follows:

Three.js takes full control of the 3D layer so that basically we can do whatever three.js can do.  We lose some functions that used to be working (will be explained below), but take advantage of great 3D capabilities directly coming from three.js. An example of rendered 3D objects is shown below:

As you can see, there are grid lines, which can be turned on and off by the following toggle switch associated with the stage:

Moreover, we can change the position of the camera using the following blocks newly added for the stage. Using this blocks, we can change the perspective of the 3D space easily.

• Functions not working anymore

So the real 3D starts working fine, but there are some drawbacks. So far, I have noticed the following functions are not working anymore with the new implementation and needs to be implemented.
      1. Drag & drop
      2. Nesting
      3. Thumbnail display
      4. Overlapping with 2D objects

• 3D functions to be implemented

Aside from the above four functions, there are some more functions essentially for 3D:
      5.  ✔ Transformation (move x y z)
      6.  ✔ Rotation (rotate n degrees in x, m in y, l in z)
      7.  ✔ Scaling
      8. Textures (apply this image to all the surface)
      9. 3D pen (render common 3D shapes such as cube, arc, sphere, cone, ... and so on)

• Plans

5 to 7 are done.  So, I will plan to work on 8 and 9 first and then 1 and 2. Considering the time remaining for the summer, maybe I can finish 8 and 9, but not sure about 1 and 2. I will do my best and let's see how far we can go.

Weeks of 6/25/2014 - 7/8/2014

We just realized a significant issue in the system we have implemented so far.

• Current Implementation

 Currently, the 3D system in C-Snap works as follows:

1. Each object on the stage renders its image on its own (invisible) canvas
2. The objects are sorted in ascending order of Z-values
3. The objects are rendered on the stage (visible canvas) by the C-Snap drawing system

From the C-Snap drawing system perspective, 2D and 3D objects are more or less the same -- They are both SpriteMorphs that have an associated canvas each, and thus 2D and 3D objects can coexist without a problem.

However, this implementation has a significant limitation: we cannot render multiple objects that are both in front of and behind of one another. Instead, all we can do is change the rendering order of canvases. This issue is depicted in the following diagram:

• Solution to the Issue

One possible solution is to let three.js do the complicated 3D job. We can give three.js a large canvas (perhaps the one that StageMorph has) and let it render multiple 3D objects on that canvas. After that, we  somehow overlay multiple canvases for  2D objects and the single canvas for 3D object.

However, this solution is going to be a major change to the existing system and there will be a lot of technical challenges including the following:

1. Integration to the existing system
2. Overlapping between 2D and 3D objects
3. Drag and drop of 3D objects

Having said that, this solution is good in terms of the completeness of 3D support, so I would like to go ahead in this direction.

Weeks of 6/11/2014 - 6/24/2014

• Work plans

After talking to Charles, we agreed to implement the following 3D functinalities:

  * Transformation (move x y z)
  * Rotation (rotate n degrees in the x, m in y, c in z)
  * Scaling (make the thing .5 times it's current size in the z axis)
  * Textures (apply this image to all the surface on the object) 
 

• Transformation

Suppose we are looking at the stage on the X-Y plane from point A at (z_eye, 0), point C at (z_obj, h_obj) is projected as point B on the stage as follows:

Add caption

We can apply the same idea on the X-Axis too. So if we can implement this idea perfectly, then object movements on the Z-axis look naturally. However, what I have implemented so far is not perfect -- when moving an object in the Z direction, I only change the size of the object, but keep the x and y coordinates the same. This is illustrated in the following example:

The center of the object does not change and is always at (x, y) = (100, 100) -- logically and physically. In this example, the viewpoint is at (x, y, z) = (0, 0, 200), so following the idea presented above, the projected X and Y coordinates on the stage should be:

• For (x, y, z) = (100, 100, -100): (x_proj, y_proj) = (66.67, 66.67)
• For (x, y, z) = (100, 100, 100) : (x_proj, y_proj) = (200, 200) 

To implement this correctly, 3D objects need to have logical coordinates as well as physical coordinates on the stage, but it causes a major change to the existing 2D system. So, it might be less problematic if we keep the 3D system as it is currently implemented.