This page provides links, help guides, and other resources for the SciServer suite of tools.
Links to SciServer Tools
The following tools are part of the current SciServer release, Altair Update 2. You can use these links to open a tool and follow along as you read the help sections below.
The guides below will get you started with the basics of our new tools. Each tool also has a help section which you can refer to while using it.
The SciServer-enabled release of CasJobs includes these new features:
- A Single Sign-on Login Portal will log you in to all SciServer resources.
- A new MyScratch shared, temporary scratch space that allows you to run queries with very large result sets, and store those results for later analysis.
- Direct output to the new SciDrive cloud storage system makes the contents of your MyDB available as files through a Dropbox-like file system interface.
Brief instructions for using these new features are given below. Use these new features as they come up naturally in the course of your work. All other features of CasJobs should be unchanged.
As always, if you encounter problems or have questions or suggestions, please let us know using our feedback form.
The first thing you may notice about CasJobs is that the login system has changed. The “create account” and “login” links have gone from the navigation menu, and have been replaced by the new SciServer login control in the top right corner of the window.
Click “Login” in the SciServer control to login to CasJobs via the SciServer Login Portal. Once logged in, you will automatically be redirected to the CasJobs Query page.
SciDrive is a new SciServer tool you can use to save your MyDB tables. In addition to downloading the contents of your MyDB tables in CSV, FITS, and VOTable formats, you can also save them to SciDrive.
The screenshot below shows how to select SciDrive from the Table Download menu. The textbox after SciDrive is the name of the container in SciDrive into which the table will be saved; the default value is
When you click Go, you will be taken to the Output page, where your request will be listed under “Pending Output”. Once the write to SciDrive finishes, the name of the table will appear under “Available Output”. The next time you log in to SciDrive, you will see the contents of that table in the container whose name you specified (defaulting into first_container). See the SciDrive instructions below to learn how to work with the data file you just wrote.
MyScratch provides support for very large queries.
The Sloan Digital Sky Survey contains so much data about the night sky that it is all too easy to write a scientifically useful query that fills up all the space in your MyDB. Although you can request additional space, it takes time for us to grant you the space, and it interrupts your work. This delay is particularly frustrating when you are selecting intermediate data that you know you will subset or summarize later, and so you know that the huge result set will only be temporary.
The new SciServer-based CasJobs system solves this problem by offering MyScratch shared temporary storage space. MyScratch is like a single very large database shared among all CasJobs users. Because MyScratch is shared among all users, older data are deleted when more space is needed for new user queries, but we guarantee that your data in MyScratch will be available for at least one week.
To select data into MyScratch, modify your query by adding an
into clause. For example, to select into a table called MyTable in the MyScratch space, add
into [myscratch:default].MyTable. Formatting is very important – the default myscratch call must be all lowercase, enclosed in brackets. The screenshot below shows an example of a query that selects SDSS DR12 photometric data into MyScratch.
Once you have selected data into a table in the MyScratch space, you can work with that table on your MyDB page just as you could any table in your own MyDB. The key is to select the [myscratch:default] context from the dropdown list at the top left of your MyDB page, just under the CasJobs logo. Once you choose the [myscratch:default] context, you can click on the Tables link to explore tables in MyScratch, just as you would in your own MyDB.
Once your query has completed, the output table you just created should appear in the tables listed for the [myscratch:default] context. Select the table you just created by clicking on its name in the tables listed in the MyScratch context. You will see the table contents displayed on the right hand side just like with MyDB tables. Note that there are fewer buttons for MyScratch table – you can add notes to the table, view a sample of the data or download it.
You can also query tables in MyScratch from the Query page, just as you can tables in your own MyDB. To query a table in MyScratch, you must do two things: select [myscratch:default] as the context from the Context menu above the query window, AND also add [myscratch:default] in front of the table name in your query, like this:
If you want to download the MyScratch table, the procedure is the same as if you wanted to download a table from your own MyDB. Click on the Download button and select an output format. SciDrive is available as an option for tables in MyScratch as well.
If your table is large, the download operation could take several minutes or even hours. It might be more efficient to view and save subsets of your MyScratch table through the Query page.
Note that you have to be in the [myscratch:default] context to do anything with the MyScratch table, and even if you are in that context, you must always explicitly prefix the table name with the
More help is available on CasJobs’s Help Page.
If you encounter bugs or have questions or suggestions, please let us know using our feedback form!
This section describes how to connect to SciServer Compute, create a container, and start up your first notebook.
- Login to SciServer:
Navigate your browser to portal.sciserver.org.
If you don’t remember your CasJobs username or password, you can create a new one.
Because your compute environment today will be on the SciServer test system, you will not be able to access your production or pre-production Casjobs.
- Start up SciServer Compute:
Navigate your browser to compute.sciserver.org.
- Create a Container:
You will need to create a container, if you don’t already have one. Click on Create Container.
Give your container a name.
Choose an image. Currently the only image available is Astronomy, which includes Python 2 and Python 3 as the base image.
Choose volumes to attach. You will always want to attach persistent and scratch. Persistent holds your working files and scripts. Scratch is temporary space for large result sets. Any data you save in Scratch is guaranteed to be available for a week, minimum.
You can also, optionally, attach the Volume SDSS DAS, which contains SDSS Data Archive Server files.
- Open your Container:
Click on its name. It contains two items (or three if you attached SDSS_DAS): persistent, and scratch. All your items should be kept in persistent, so click on persistent to open it.
- Create or import your notebook:
You can now get started with your Notebook. Please have a look at our example notebook walkthrough to see what you need to do to connect to CasJobs, your MyDB, and SciDrive.
More help is available on Compute’s Help Pages.
SciServer hosted a Compute Workshop in April 2016. Example notebooks from the workshop are available on the workshop documentation pages.
If you encounter bugs or have questions or suggestions, please let us know using our feedback form!
When you first arrive at the login portal, you will see this login form:
If you already have a CasJobs username and password, enter it here. Be sure to enter your username exactly as it is on the current CasJobs site. If you have forgotten your password, you can reset it on the page, or email us at firstname.lastname@example.org to request a new one.
If you do not already have an existing CasJobs account, click “Register New Account.” The login control will change to look like this:
Pick a username, enter your email, and enter a password at least 5 characters long. Confirm your password, check the checkbox, and click Submit.
Using SciServer Tools
Once you are logged in, your view of the portal will change to what you see below.
The links by “SciServer Tools” will lead you to SkyServer, CasJobs, and SciDrive. From the logged-in view of the portal, you can also change your password, send us an email, or log out.
If you forget your password at any time, you can reset it by clicking the “Reset Password” link. A new password will be sent to you by email.
Please log in or register, and let us know any problems you encounter through our feedback form.
SciDrive is a new tool from SciServer that provides a cloud-based file system integrated with MyDB. SciDrive allows you to convert smoothly from the database-centric view of SkyServer/CasJobs to your own local file system.
When you first go to SciDrive, you will see the introductory page with a login button and some instructions (screenshot below). You will see this whether you have logged in to the Portal already or not.
If you are already logged into the Portal and you press the Login button, you will be taken directly to the SciDrive application. If you are not logged into the Portal and you press Login, you will be taken to the Portal first to log in, then automatically redirected to the SciDrive application.
SciDrive has a Help button in the top right with basic instructions on its use. We are adding more information to this help section throughout our testing phase; your feedback as an early adopter will help us figure out how to best support SciDrive users.
Once you are logged in to SciDrive, you will first see the application dashboard. The action buttons at the top let you perform various actions in the SciDrive interface – mouse over each button for a label. Below is a list of your data containers; if you are new to SciDrive, you will have only the default, called “first_container”. Containers are directories where you can put files. You will also see a small tree-view icon in the top left, that says “Root” next to it. The tree view shows the current hierarchy view. You can only create new containers/folders in “Root”; you cannot add files directly to root.
Click on the container name (e.g. first_container) to see the contents of that container. The screenshot below shows the application home page with first_container selected and two previously-loaded files. If you have not yet loaded any files, it will instead say “No items to display”.
To upload files into SciDrive, simply drag the files into the SciDrive window. Depending on the file size, it may take a moment to process the file. Once it is processed, the page will show the size and type of the file(s). You can upload any type of file to SciDrive, but the main science support features work best with CSV files.
After you have uploaded files into SciDrive, you can mouseover the file names for additional options. The new action buttons that appear allow you to share the file, to download or delete it, or to see file metadata (under development).
Metadata Extraction (Under Development)
If you will be using a folder to drop in files that you would want to be parsed and copied to MyDB table, you first need to set up the appropriate plugin to read file metadata. This can be done with the SciDrive application by selecting the container for which you want to enable metadata extraction.
With the container selected, click the Settings icon in the top row of action buttons – the one that looks like a gear, as shown here:
Clicking the Settings icon will open up a new dialog box that looks like this:
Go to the “Metadata Extractors” tab (it should be selected by default), then select “CasJobs/MyDB” from the left-hand menu (again, it should be selected by default). Set the switch to ON, check any containers you would like to enable, then click Save and close the dialog box.
After enabling metadata extraction, the contents any properly-formatted CSV files you upload into SciDrive will automatically appear in your MyDB in CasJobs.
More help is available on SciDrive’s Help Page (you must be logged in to view it).
When you first arrive at SkyServer website, you will see the new login control at the top right of the window. If you are not yet logged in, the control will show a login button. If you are logged in, it will show your username.
Whether you are logged in or not, you can continue to use SkyServer as you always have. If you are logged in, new features will be available to you.
The first new feature is your History page, linked at the far right end of the SkyServer navigation menu.
Click on the History tab on the main menu will open the “My SkyServer History” page, which will show results from SkyServer tools you have used in the past. Until you have created an account and queried SkyServer while logged in, this page will be blank. Otherwise, you may see something like the screenshot below.
The page will display queries you submitted through the SQL Search tool, and also queries automatically generated by tools such as the Explore tool. The columns in this table are:
- Time: The date and time at which the query was submitted
- SkyServer Tool: The tool from which you submitted the query. This is a link, which you can click on to rerun the query.
- Request Parameters: The options you selected for the query, such as the object IDs or the query string (formatted as a JSON string).
Clicking the tool link will run the query again, showing you either the query results (for queries you submitted through SQL Search) or the original tool view (for queries generated by SkyServer tools).
More help is available on SkyServer’s Help Page.
If you encounter problems or unexpected behaviors, or if you have suggestions, please let us know through our feedback form.
Web Services for CasJobs are implemented with a RESTful API for better integration with SciServer Tools. SOAP web services are no longer supported.
Please see the CasJobs REST API specifications for information on how to use Web Services.