Skip to main content

SketchBio User Interface Controls

I’ll start by saying this: SketchBio’s user interface is still a work in progress and may change from version to version as we improve our understanding of the best way to make use of the device.

Updated for version 0.0.10 on 7/23/2014.  Changes since  version 0.0.8 in bold. New section for the menu bar options added at the bottom of the page.

We use the Razer Hydra tracker input device as our primary input.  Note: if you are on Windows, do not install the official device driver software from Razer, it conflicts with the open source drivers we use which are installed with SketchBio.

Terminology:

  • The Razer hydra has two handhelds “trackers” referred to as the left or right tracker respectively.
  • Trigger – the input device with the marking LT or RT that is furthest forward.  This is an analog device, it can tell how hard it is being pulled
  • Bumper – the button directly above the Trigger on each hand
  • Oblong button – the button between the buttons marked 1 and 2 on each hand

General Control Layout:

The general layout assumes a right handed user.  If you are left-handed, then simply switch which tracker is in which hand after the device calibrates and the program has started.  In the following I will assume a right-handed user, please just reverse the handedness of the control if you have the trackers switched.

In general, the left hand controls world operations such as zoom level or disabling collision detection, while the right hand controls “object” operations such as duplicate or set keyframe.

Left Hand Controls:

1 button – change modes.  Modes affect what the buttons on the right hand and the left hand’s bumper and trigger do and are displayed at the bottom left of the screen.  Currently this cycles between three modes (object, spring, color, and animation– see below).

2 button – enable/disable collision tests

3 button – enable/disable world physics (currently only attractive forces “springs”)

4 button – zoom.  Hold this button and move the trackers closer together or further apart to zoom in/out.

Oblong button – Redo

Hi-Hat joystick – move the camera independently of the workspace of the trackers (look around without moving).

Right Hand Controls:

The only right hand controls that are the same across all modes is:

Hi-Hat joystick – move left and right to move backward or forward (respectively) in the animation timeline.

Hi-Hat joystick – move up and down to change level of selection within a group.  Move up to select an object within the currently selected group (just move it up and back to center).  Move down and back to center to change from selecting an object within the group to selecting the whole group again.

Oblong button – Undo

Objects Mode:

In this mode the left and right bumper buttons can be used to grab objects when the tracker is close enough to them (indicated by a bounding box appearing around the object).  Note that you must be close enough in all three axes including depth into the screen.  This is why we added the following: when a tracker is not close enough to any object, pressing the bumper button grabs and moves the whole world (allowing you to move the object to you).  Zooming helps with this sort of thing too.

For these buttons, when the tracker is close enough to the object I will refer to the object as being “selected”

Right Hand Controls:

1 button – replicate object (create from one base object, a self-symmetric strand such as an actin fiber from a single g-actin).  Move so that the right hand is selecting the object, then press the button. To select the number of copies, move the left hand so that it is highlighting the base object, then pull the left trigger  to increase the number of copies (careful, it is an exponential scale, pulling twice as far results in the square of the number of copies).  When the desired number of copies is shown, move the left hand outside the object so that it is no longer highlighted to finalize them. In order to move the second in object in relation to the base object (and have the others follow by example), switch to groups mode and flick the right joystick down. You should now be able to select the base and second object individually and rearrange them to create a crystal-by-example structure.

2 button – lock transforms (ensure that the transformation between objects a and b is and remains exactly equal to the transform between objects c and d).  Press the button while the right tracker is selecting object a, hold it and move so that the right hand is selecting object c, then release.  Then move so that the right hand is selecting object b and press the 2 button again, hold and move so that object d is selected and release.  This can be used in combination with the replication feature from above to ensure that two actin fibers are structured exactly the same way or alone to make sure that two vinculins are attached to an actin fiber in exactly the same way relative to different monomers in the fiber.  The original use case was to ensure that two chains of fibrinogen were exactly the same while they were wrapped around each other.

3 button – set transforms (input the transformation between objects A and B in a dialog).  Press the button, move the right tracker so that object A is selected and release.  Then press the button, move the right tracker so that object B is selected and release.  Then a dialog will come up in which you can input the position and orientation of B relative to A.  Note that this will move B and will not move A when you click OK in the dialog.

4 button –  this button is delete object.  It will delete whatever object is selected when it is released.

Springs Mode:

This mode is for editing attractive forces between objects or “springs” (the equation used is slightly different from the spring equation however).  Each “spring” has two endpoints that can either be inside an object (attached to the object) or fixed in space.  If both ends are fixed in space, the “spring” does nothing.

Instead of grabbing and moving objects, in this mode the left and right bumper buttons are used to grab the endpoints of springs.  The springs will indicate that they are selected for grabbing by displaying a bounding pyramid around the endpoint that is selected.  Press the corresponding bumper button and hold to grab the spring’s end.  You can drag it around and drop it either inside an object (in which case it becomes attached to the object at that point), or out at a location (it becomes fixed to that location). While a spring is grabbed, you can adjust its rest length by squeezing the left trigger – squeezing it further makes the rest length a larger fraction of its current length.

Right Hand Controls:

1 button – Delete Spring.  Deletes whichever spring is highlighted by the right tracker.

2 button Snap connector to X-terminus.  This will snap the selected connector to the N-terminus of the molecule it is currently attached to.  If you wish to snap the connector to the C-terminus instead, press the right trigger while holding down the button and release the button with the trigger held down.

3 button – create spring.  Creates a new spring with both endpoints at the current location.  If the button is pressed and held, then the left trigger can be used to control the stiffness of the created spring.  The range goes from the default (fairly stiff, trigger is not pressed at all) to a spring with spring constant of zero (trigger held all the way).

4 button – Create transparent connector.  This is not a spring, although it can connect the same things a springs.  It applies no forces to objects, but when rendered in Blender these will show up where normal springs will not.

Animations Mode:

This mode is for editing keyframes (object time/position combinations) and properties that are only applied during an animation.

The left and right bumper buttons behave exactly the same as Objects Mode.

Right Hand Controls:

1 button – Set a keyframe on every object in the scene at the current time.

2 button – Set a keyframe for the current location, orientation, and time on the object

3 button – create camera.  This is for animations.  It creates a camera object that can be positioned to set the viewpoint for animation.

4 button – Previews the animation from the perspective of the camera (see place camera in Objects mode).  Pressing this again during the preview cancels the preview.

Colors Mode:

This mode is for modifying attributes of individual objects (currently just color).

The left and right bumper buttons behave exactly the same as Objects Mode.

Right Hand Controls:

1 button – Cycle through color maps on the selected object or connector (mostly solid colors but one is a blue-to-red scale)

2 button – Change what data is used to color the selected object (right now toggles between solid color, coloring by position along the chain and coloring by surface charge)

3 button – Toggle object visible during animation

4 button – Toggles whether “invisible” objects are shown.  This is a preview mode so that you can see what will be visible during the animation.  It is not recommended to work in this mode as invisible objects still collide with other objects.

Groups Mode:

This mode is for modifying the grouping of objects.  Copy and paste went here too, but only because I was out of button in other modes.

The left and right bumper buttons behave exactly the same as Objects Mode.

Right Hand Controls:

1 button – Group or ungroup objects.  To group two objects, select one with the left hand and one with the right hand and press the button.  To ungroup, select the group as a whole with the left hand and the object to remove with the right hand (objects within groups will highlight with red outlines when selected) and press the button.  You do not need to do level of selection modification to ungroup objects.

2 button – Copy current object.  This copies whatever object is selected when the button is released.

3 button – Nothing yet

4 button – Paste whatever is copied (if it is a valid thing to paste)

Menu Bar

The following controls do not use the Razer Hydra, and can be found in the menu bar at the top of the SketchBio window.

File Menu:

  • Open – Add an object to your project by opening one of the following file formats:
    • OBJ file
    • PDB file (one already saved on your computer; use [File->Import->PDB_ID] to import from the Protein Data Bank)
    • VTK file – The file is assumed to be one of the VTK files generated by a previous SketchBio project, and if filenames matching other VTK files that SketchBio would have generated with it are found, they are included in the resulting model.
    • ZIP Archive (use this to load a structure you had previously saved using [File->Save Copied Structure])
  • Import – import an object to add to your project from:
    • PDB_ID – Import a structure from the Protein Data Bank using its PDB ID
  • Export
    • Blender Animation – Render a video (AVI) animation from your project
    • Florosim Simultation – Export the current state of the project as a Florosim simulation
  • Filter
    • Simplify – Simplify an OBJ file
  • Save Project – Save the current project
  • Save Project As… – Save the current project in a new directory, not affecting the save state of the original project directory
  • Load Project – Open a different SketchBio project
  • Save Copied Structure – Save a grouped structure from SketchBio into a ZIP archive. This structure can then be loaded into your other SketchBio projects so you don’t have to construct it by hand every time.

Insert Menu:

  • Ellipsoid – Add an ellipsoid to the project by specifying its diameter in each dimension
  • Measuring Tape – Add a tape that functions just like transparent connectors, except it displays its length on screen. Use this to measure distances between objects – it can be snapped to the C-terminus or N-terminus of a protein just like other connectors
  • Helix – Add a helical guide, which does not interact with the physics of the scene. This item can help the user create helical crystal-by-example structures by providing a visual guide. User can specify the radius of the helix, the number of turns, and the length of each turn along the axis.
  • Ring – Add a torus by specifying a ring radius (the distance from the center of the ring to the center of the tube) and the cross section radius (the radius of the tube that forms the ring).

View Menu:

  • Set View Time – Enter the time you want to go to in your project. You can also move through time with the Razer Hydra’s right thumbstick, but this option allows for more precise navigation
  • Set Luminance – In crystal-by-example structures, the luminance of individual copies varies randomly to help distinguish individual subunits within a structure. Use this option to set the minimum luminance level (between 0 and 1) for these structures, with 0 corresponding to the greatest visual conrast and 1 making all subunits display at maximum luminance instead of varying.
  • Create Camera for Viewpoint – Add a new camera corresponding to your current view of the project
  • Set Camera to Viewpoint – Set the existing camera to your current view of the project
  • View in Full Resolution – Select which objects you want to have displayed in their full resolution. Note: we cannot display all objects in the project at full resolution because this slows down the rendering speed to an unusable level, given enough objects.
    • Grabbed Objects
    • Nearby Objects – Objects near any grabbed objects (which could potentially collide with it) will be displayed in full resolution

Options Menu:

  • Collision Mode: Select the type of collision response
    • Pose Mode 1 – Only objects currently being manipulated will move. An object you are grabbing will not push around other objects.
    • Old Style – Standard rigid-body dynamics. The manipulated object can push other objects around.
    • Binary Collision Search – If there is a collision, back off by 1/2 the force/torque.
    • Pose Mode PCA – Only objects currently being manipulated will move. An object you are grabbing will not push around other objects.
  • Grab Mode: Select the method for moving objects when they are grabbed. Use whichever one feels more comfortable for you.
    • Direct mapping – Motion of your hand is directly mapped to the motion of the object; object moves exactly in time with your hand.
    • Force-based – Moves objects using forces by attaching invisible springs from the tracker to the object in order to pull it around.
  • World Springs On – Toggle spring forces on/off
  • Collision Tests On – Toggle collision detection on/off
  • Reset Device – Restarts the internal VRPN server. Try this if your input device is not behaving correctly (e.g., the tracker object does not move in the same direction as your hand when using the Razer Hydra).