Getting Started
In this demonstration, we will show you how to setup and run a basic simulation using Sorcha.
See also
Sorcha is designed to allow multiple instances to be run in parallel in order to accommodate simulations with very large numbers of synthetic planetesimals by breaking up the job across multiple live processes. We recommend first starting with the examples below, before moving on to our parallel processing/high-performance computing (HPC) guide.
Grab All The Demo Files
Use the following command to copy all the demo inputs and configuration file to your local directory:
sorcha demo prepare
Note
All input data files in this example are white-space separated format solely for the ease of reading.
Note
The optional -p (--path) flag allows you to specify a specific location to copy the demo input files. The demo command you will be using expects these files will be in your current workingn directory.
Tip
If the files already exist and you want a fresh copy, the -f (--force) flag can be used to force a fresh copy of the files to be generated.
Orbit and Physical Parameters Files For the Input Solar System Small Body Population
Orbit File
The Orbit File in the demo set is labeled sspp_testset_orbits.des, which contains the orbits of five synthetic objects:
ObjID a e inc node argPeri ma epochMJD_TDB FORMAT
2010_TU149 2.2103867 0.8245336 1.9679 58.91911 92.60419 324.21145 60200 KEP
2010_TC209 2.6411033 0.2378397 17.51668 25.01678 18.48118 347.09673 60200 KEP
2011_OB60 103.711231 0.6461554 19.42877 142.66859 226.19366 358.36581 60200 KEP
2011_WJ157 99.2799891 0.6226317 27.06784 76.41243 58.43528 355.56379 60200 KEP
2011_YA42 3.1910723 0.1754333 17.90056 291.08053 99.36952 70.55556 60200 KEP
2012_AW12 1.6899997 0.6063531 51.51998 151.77178 246.62641 150.38051 60200 KEP
2012_BB14 1.0627063 0.098754 2.64121 316.78101 255.55235 135.8483 60200 KEP
2012_HW1 1.0607726 0.6615131 49.57978 207.81039 163.70136 63.97462 60200 KEP
2013_CC263 3.0000262 0.1126188 8.97766 336.5147 98.64025 69.02504 60200 KEP
2013_GD138 43.6879964 0.1086886 5.43381 57.00856 146.16056 24.30927 60200 KEP
Physical Parameters File
The Physical Parameters File is named sspp_testset_colours.txt. This file assigns colors, phase curve properties, and absolute magnitudes to each of the simulated small bodies whose orbits are defined in our input orbit file. The contents of the file are below:
ObjID H_r GS u-r g-r i-r z-r y-r
2010_TU149 20.7 0.15 2.13 0.65 -0.19 0.14 -0.14
2010_TC209 18.45 0.15 1.9 0.58 -0.21 -0.3 -0.39
2011_OB60 6.9 0.15 1.72 0.48 -0.11 -0.12 -0.12
2011_WJ157 4.92 0.15 2.55 0.92 -0.38 -0.59 -0.70
2011_YA42 16.36 0.15 2.13 0.65 -0.19 0.14 -0.14
2012_AW12 19.79 0.15 1.9 0.58 -0.21 -0.3 -0.39
2012_BB14 24.99 0.15 1.72 0.48 -0.11 -0.12 -0.12
2012_HW1 19.99 0.15 2.13 0.65 -0.19 0.14 -0.14
2013_CC263 18.46 0.15 1.72 0.48 -0.11 -0.12 -0.12
2013_GD138 8.14 0.15 2.55 0.92 -0.38 -0.59 -0.70
Note
For this demo, we have defined chosen the main filter to be r-band.
Survey Pointing Database
We'll be using the first year of the baseline v2.0 rubin_sim LSST cadence simulation as the Survey Pointing Database (baseline_v2.0_1yr.db).
Configuration File
The sorcha_config_demo.ini Configuration File will initalize Sorcha. The contents of the file are below:
# Sorcha Configuration File
# This configuration file is made for use with the demo files that come packaged
# with Sorcha. We do not necessarily recommend you use this for your own science.
# Instead, run sorcha init on the command line to copy suitable example config files into
# a directory of your choice.
[INPUT]
# The simulation used for the ephemeris input.
# ar=ASSIST+REBOUND interal ephemeris generation
# external=providing an external input file from the command line
# Options: "ar", "external"
ephemerides_type = ar
# Format for ephemeris simulation input file if a file is specified at the command line.
# This is also the format to which ephemeris files will be written out, if specified.
# Options: csv, whitespace, hdf5
eph_format = csv
# Sorcha chunk size: how many objects should be processed at once?
size_serial_chunk = 5000
# Format for the orbit, physical parameters, and complex physical parameters input files.
# Options: csv or whitespace
aux_format = whitespace
# SQL query for extracting data from the pointing database.
pointing_sql_query = SELECT observationId, observationStartMJD as observationStartMJD_TAI, visitTime, visitExposureTime, filter, seeingFwhmGeom as seeingFwhmGeom_arcsec, seeingFwhmEff as seeingFwhmEff_arcsec, fiveSigmaDepth as fieldFiveSigmaDepth_mag , fieldRA as fieldRA_deg, fieldDec as fieldDec_deg, rotSkyPos as fieldRotSkyPos_deg FROM observations order by observationId
[SIMULATION]
# Configs for running the ASSIST+REBOUND ephemerides generator.
# the field of view of our search field, in degrees
ar_ang_fov = 2.06
# the buffer zone around the field of view we want to include, in degrees
ar_fov_buffer = 0.2
# the "picket" is our imprecise discretization of time that allows us to move progress
# our simulations forward without getting too granular when we don't have to.
# the unit is number of days.
ar_picket = 1
# the obscode is the MPC observatory code for the provided telescope.
ar_obs_code = X05
# the order of healpix which we will use for the healpy portions of the code.
# the nside is equivalent to 2**ar_healpix_order
ar_healpix_order = 6
[FILTERS]
# Filters of the observations you are interested in, comma-separated.
# Your physical parameters file must have H calculated in one of these filters
# and colour offset columns defined relative to that filter.
observing_filters = r,g,i,z,u,y
[SATURATION]
# Upper magnitude limit on sources that will overfill the detector pixels/have
# counts above the non-linearity regime of the pixels where one can’t do
# photometry. Objects brighter than this limit (in magnitude) will be cut.
# Comment out for no saturation limit.
# Two formats are accepted:
# Single float: applies same saturation limit to observations in all filters.
# Comma-separated list of floats: applies saturation limit per filter, in order as
# given in observing_filters keyword.
bright_limit = 16.0
[PHASECURVES]
# The phase function used to calculate apparent magnitude. The physical parameters input
# file must contain the columns needed to calculate the phase function.
# Options: HG, HG1G2, HG12, linear, none.
phase_function = HG
[FOV]
# Choose between circular or actual camera footprint, including chip gaps.
# Options: circle, footprint.
camera_model = footprint
# Path to camera footprint file. Uncomment to provide a path to the desired camera
# detector configuration file if not using the default built-in LSSTCam detector
# configuration for the actual camera footprint.
# footprint_path= ./data/detectors_corners.csv
# Fraction of detector surface area which contains CCD -- simulates chip gaps
# for OIF output. Comment out if using camera footprint.
# Default: 0.9.
# fill_factor = 0.9
# Radius of the circle for a circular footprint (in degrees). Float.
# Comment out or do not include if using footprint camera model.
# circle_radius = 1.75
# The distance from the edge of a detector (in arcseconds on the focal plane)
# at which we will not correctly extract an object. By default this is 10px or 2 arcseconds.
# Comment out or do not include if not using footprint camera model.
footprint_edge_threshold = 2.
[FADINGFUNCTION]
# Uses the fading function as outlined in
# Chesley and Vereš (2017) to remove observations.
# Width parameter for fading function. Should be greater than zero and less than 0.5.
# Suggested value is 0.1 after Chesley and Vereš (2017).
fading_function_width = 0.1
# Peak efficiency for the fading function, called the 'fill factor' in Chesley and Veres (2017).
# Suggested value is 1. Do not change this unless you are sure of what you are doing.
fading_function_peak_efficiency = 1.
[LINKINGFILTER]
# Remove this section if you do not wish to run the SSP linking filter.
# SSP detection efficiency. Which fraction of the observations of an object will
# the automated solar system processing pipeline successfully link? Float.
SSP_detection_efficiency = 0.95
# Length of tracklets. How many observations of an object during one night are
# required to produce a valid tracklet?
SSP_number_observations = 2
# Minimum separation (in arcsec) between two observations of an object required
# for the linking software to distinguish them as separate and therefore as a valid tracklet.
SSP_separation_threshold = 0.5
# Maximum time separation (in days) between subsequent observations in a tracklet.
# Default is 0.0625 days (90mins).
SSP_maximum_time = 0.0625
# Number of tracklets for detection. How many tracklets are required to classify
# an object as detected?
SSP_number_tracklets = 3
# The number of tracklets defined above must occur in <= this number of days to
# constitute a complete track/detection.
SSP_track_window = 15
# The time in UTC at which it is noon at the observatory location (in standard time).
# For the LSST, 12pm Chile Standard Time is 4pm UTC.
SSP_night_start_utc = 16.0
[OUTPUT]
# Output format of the output file[s]
# Options: csv, sqlite3, hdf5
output_format = csv
# Controls which columns are in the output files.
# Options are "basic" and "all", which returns all columns.
output_columns = basic
[LIGHTCURVE]
# The unique name of the lightcurve model to use. Defined in the ``name_id`` method
# of the subclasses of AbstractLightCurve. If not none, the complex physical parameters
# file must be specified at the command line.lc_model = none
lc_model = none
[ACTIVITY]
# The unique name of the actvity model to use. Defined in the ``name_id`` method
# of the subclasses of AbstractCometaryActivity. If not none, a complex physical parameters
# file must be specified at the command line.
comet_activity = none
Note
The demo configuration file sets the output to be in CSV (comma-separated values) format.
Downloading Auxiliary Files For the Ephemeris Generator
To run Sorcha's built in ephemeris generator, you will need to download the auxiliary files required by assist and rebound for performing the N-body integrations.
To install the necessary SPICE (Spacecraft, Planet, Instrument, C-matrix, Events) auxiliary files and other required data files for ephemeris generation (774 MB total in size) run the following command on the command line:
sorcha bootstrap
Note
This script will download and store the auxiliary files in your computer's local cache directory by default which is where the demo command will expect the files are installed. The optional --cache flag allows you to specify a specific location to download the auxiliary files. If the files have already downloaded and want a fresh download, you need to use the -f flag.
Warning
These files can change/be updated with the revised positions of the planets every once in a while. So if you're running simulations for population statistics, we recommend downloading these files to a directory and having all Sorcha runs these files for consistency.
Tip
If the auxiliary files are installed in a different location you will need to specify their location using the --ar flag when running Sorcha
Running Sorcha
We now have all the required input files. If you downloaded the Sorcha repository, start by moving into the Sorcha directory or make a demo directory called demo and move/copy all the input files into there. For this example run, we assume that you have downloaded the required ephemeris generator's auxiliary files to ./ar_files. Check the Installation instructions for further details.
Next, let's take a look at the command line arguments for the sorcha run. On the command line, typing:
sorcha run --help
will produce
usage: sorcha run [-h] -c C -o O --ob OB -p P --pd PD [--er ER] [--ew EW]
[--ar AR] [--cp CP] [-f] [-s S] [-t T] [-l] [--st ST]
Run a simulation.
options:
-h, --help show this help message and exit
Required arguments:
-c C, --config C Input configuration file name (default: None)
-o O, --outfile O Path to store output and logs. (default: None)
--ob OB, --orbits OB Orbit catalog file name (default: None)
-p P, --physical-parameters P
Catalog of object physical parameters (default: None)
--pd PD, --pointing-db PD
Survey pointing information (default: None)
Optional arguments:
--er ER, --ephem-read ER
Previously generated ephemeris simulation file name,
required if ephemerides_type in config file is
'external'. (default: None)
--ew EW, --ephem-write EW
Output file name for newly generated ephemeris
simulation, required if ephemerides_type in config
file is not 'external'. (default: None)
--ar AR, --ar-data-path AR
Directory path where Assist+Rebound data files where
stored when running bootstrap_sorcha_data_files from
the command line. (default: None)
--cp CP, --complex-physical-parameters CP
Catalog of object complex physical parameters
(default: None)
-f, --force Force deletion/overwrite of existing output file(s).
Default False. (default: False)
-s S, --survey S Survey to simulate (default: rubin_sim)
-t T, --stem T Output file name stem. (default: SSPPOutput)
-l, --log-level Print additional information to log while running
(default: True)
--st ST, --stats ST Output summary statistics table to this stem filename.
(default: None)
Now that you know how to provide the input files, let's go run a simulation: You can find the command to run the Sorcha demo on the command line in two ways. First on the command line:
sorcha demo howto
Or you can in an interactive python session or jupyter notebook. You can run the following
from sorcha.utilities.sorcha_demo_command import get_demo_command
print(get_demo_command())
sorcha run -c sorcha_config_demo.ini -p sspp_testset_colours.txt --orbits sspp_testset_orbits.des --pointing-db baseline_v2.0_1yr.db -o ./ -t testrun_e2e --stats testrun_stats
Four files will be created in the current directory after you run the sorcha run command. Two will be log files (.log and .err). The other two files will be CSV (comma-separated values; .csv) files.
Tip
The log files have One *.log with information about the run and one *.err that is used to save error messages from the run. The *.err log file should be empty if Sorcha ran successfully.
One of the files will be the detections file (information about each observation of the input small body population in the simulated astronomical survey) will appear in a CSV file (testrun_e2e.csv) in your current directory. The first 21 lines of this CSV file should look like this (because of the random number generation the values will not be exactly the same):
ObjID,fieldMJD_TAI,fieldRA_deg,fieldDec_deg,RA_deg,Dec_deg,astrometricSigma_deg,optFilter,trailedSourceMag,trailedSourceMagSigma,fiveSigmaDepth_mag,phase_deg,Range_LTC_km,RangeRate_LTC_km_s,Obj_Sun_LTC_km
2011_OB60,60225.247167832895,2.8340797698367206,-12.194064864430457,1.982568043537185,-11.89548832444173,7.486711256659074e-06,i,22.376594553193442,0.06355310419223806,23.811384411034556,0.5514796131072949,5381400132.572375,8.911895028240309,5521397111.153805
2011_OB60,60225.27094733885,2.8340797698367206,-12.194064864430457,1.9820904884532524,-11.895657418876818,1.970228013143965e-05,z,22.53444711068445,0.1396300773217608,22.88254562089191,0.5519774373107185,5381418504.312298,8.9707869959089,5521396245.784249
2011_OB60,60225.28270233429,2.8340797698367206,-12.194064864430457,1.9818347508034435,-11.895773573538131,1.368449919373666e-05,z,22.407741434577577,0.11795137669245162,23.080846371946343,0.5522233481994628,5381427629.198108,8.99782119982069,5521395818.016225
2011_OB60,60227.24471872862,1.830365601304555,-10.653419743409385,1.9444038904110243,-11.911526040922363,1.2466492192791209e-05,r,22.54790490167689,0.07307916451539992,23.764317867665344,0.5935019031571788,5382985829.821242,9.890004805584873,5521324456.507915
2011_OB60,60227.26843110208,1.830365601304555,-10.653419743409385,1.9439589118820777,-11.911731444751968,1.0506087754734058e-05,i,22.658108379468125,0.07951187390559358,23.55633186666245,0.5940086408462588,5383006152.782566,9.947927989892335,5521323594.479625
2011_OB60,60228.23229500716,1.5853750239774231,-12.795350891587582,1.925742565120629,-11.919219188556651,3.86500480773163e-05,y,22.496110087408628,0.2672681862645966,22.11010833815787,0.6145426774489476,5383831334.0043545,10.346425806177027,5521288564.603915
2011_OB60,60229.23294578834,1.7408953022743148,-11.367447303472217,1.9068734895353145,-11.926897515768195,6.448960302837916e-06,g,23.12518426360586,0.050129668307075895,24.689706821972628,0.6359862246243226,5384729639.820483,10.83405866005185,5521252216.560386
2011_OB60,60229.25681096731,1.7408953022743148,-11.367447303472217,1.9064268634084556,-11.927079044347407,1.1882851203406289e-05,r,22.700989309653675,0.0716467533855952,23.793181284412043,0.6365024755808489,5384752041.250222,10.893584442167844,5521251349.91172
2011_OB60,60230.18746404967,2.7692539237126224,-11.336818754683701,1.8890458209938288,-11.934062017945095,5.321102892399155e-06,r,22.655177235098034,0.04388740106269964,24.36853846041687,0.6565119205880392,5385624478.812854,11.19259941059988,5521217562.005192
2010_TU149,60230.2013547434,15.887030292262942,2.610528423589063,16.29087395374347,3.3663261895746817,3.3176361374860164e-06,r,21.078499119128235,0.017595597501214887,24.304131138512137,2.705921241205189,95234455.08949262,-19.599087900322974,244356465.57558417
2011_OB60,60230.21124082722,2.7692539237126224,-11.336818754683701,1.8886139473802928,-11.934253770906158,7.4281493616562675e-06,i,22.356386060833977,0.06317500696642635,23.832613354508425,0.657028918755899,5385647540.16729,11.259232111736893,5521216699.021393
2010_TU149,60230.22511697796,15.887030292262942,2.610528423589063,16.27320192134664,3.358875816706462,3.9338342269700905e-06,i,20.889811580077854,0.023711054681638352,23.680810539786222,2.72399441659693,95194288.60275628,-19.527216320203248,244312929.41427946
2011_OB60,60232.03908176997,1.5110164143244003,-10.722867861970427,1.8548461509191725,-11.947533291086549,1.9938983414143295e-05,g,22.949694681072152,0.09301679229204905,23.977465735522014,0.6964187522382299,5387468765.18048,11.729784597499286,5521150387.206843
2010_TU149,60232.16553098904,15.612829800611069,3.133869551969822,14.823742228034803,2.731810232268307,6.328177318443793e-06,g,21.729062117308526,0.030220874706558994,24.09764301693548,4.519267478947488,91989767.7256791,-18.608232614336075,240739724.22611833
2010_TU149,60232.20999081624,15.612829800611069,3.133869551969822,14.788457805912856,2.7169897051920437,4.295932727799567e-06,r,21.08552700943837,0.022413006148227464,23.89153067640296,4.566320457704941,91918544.76155414,-18.471764006054336,240657432.73435733
2010_TU149,60232.255388873135,12.89013978971145,2.6516260936487535,14.75242543901038,2.701845958731931,1.823913380280419e-05,y,21.267633716985472,0.14050435180810056,21.666965142362482,4.614513540497993,91846348.61664788,-18.33982059348594,240573385.58435732
2010_TU149,60232.25628424726,15.612829800611069,3.133869551969822,14.751738506194272,2.7015202686297477,1.2484587487147977e-05,y,20.898782168711065,0.09783634212044773,22.09608324948633,4.6154658301759115,91844928.8390283,-18.33735214676861,240571726.58441705
2011_OB60,60234.13285997302,359.8967910411155,-12.396380059574478,1.816722278843372,-11.962132423323693,2.3498482584212304e-05,g,23.07045188547079,0.12689919400757577,23.62061498969283,0.741583738945773,5389719563.883179,12.934069000905469,5521074505.936933
2011_OB60,60234.133324049544,2.781692525280924,-12.375423100526417,1.8167093393792584,-11.962119673333154,1.3435878659283135e-05,g,23.098203624627587,0.07333346456075875,24.260363722565916,0.7415938631020172,5389720083.55069,12.935436841635658,5521074489.094023
2010_TU149,60234.1415233523,12.660652540934704,1.7700691849902304,13.251308841573461,2.054362781198338,5.605865932163211e-06,g,21.740321800049845,0.02865041694935682,24.218663928769995,6.713317132331069,88908847.56113863,-17.556122562348435,237063951.55333534
The last file outputted will be the statistics or tally file (testrun_stats.csv) in CSV format that has summary statistics about each of the input objects detected by the simulated survey. The first 15 lines of this CSV file should look like this (because of the random number generation the values will not be exactly the same):
ObjID,optFilter,number_obs,min_apparent_mag,max_apparent_mag,median_apparent_mag,min_phase,max_phase,date_linked_MJD
2010_TC209,g,1,21.01487743175703,21.01487743175703,21.01487743175703,4.822674862328414,4.822674862328414,60259.0
2010_TC209,i,2,20.233601443479177,20.402532052762812,20.318066748120994,4.902659658311544,7.356710425955389,60259.0
2010_TC209,r,2,20.490376193425977,20.633063031128895,20.561719612277436,4.821141532804077,7.345792889497847,60259.0
2010_TC209,u,0,,,,,,
2010_TC209,y,0,,,,,,
2010_TC209,z,1,20.091469460234205,20.091469460234205,20.091469460234205,4.899566415675729,4.899566415675729,60259.0
2010_TU149,g,2,21.729062117308526,21.740321800049845,21.734691958679186,4.519267478947488,6.713317132331069,60234.0
2010_TU149,i,3,20.864877364325775,20.93846151886477,20.889811580077854,2.72399441659693,24.70404579388877,60234.0
2010_TU149,r,3,21.078499119128235,21.08552700943837,21.07893275938656,2.705921241205189,7.9060764137927375,60234.0
2010_TU149,u,0,,,,,,
2010_TU149,y,4,20.898782168711065,21.267633716985472,21.077290657613517,4.614513540497993,14.447194538988198,60234.0
2010_TU149,z,3,21.065476296172363,21.33938121692864,21.25570051003638,13.051242279034328,24.76913955178594,60234.0
2011_OB60,g,7,22.949694681072152,23.242615022852355,23.098203624627587,0.3785543357245827,1.2668113261396663,60228.0
2011_OB60,i,20,22.127245775286116,22.794409797452477,22.452544099421793,0.3324387390222129,1.5791369649153313,60228.0
Tip
If you want to run this command a second time you'll need to add a -f flag to the command line to force overwriting output files that already were exist in the output directory. Do note that the previous run's log and error log files will not be removed. New log files are generated at each run.
Available Commands Within Sorcha
To see all the commands/utilities available within Sorcha, run:
sorcha --help
usage: sorcha [-h] [--version] [{bootstrap,cite,demo,init,outputs,run}] ...
Sorcha survey simulator suite.
positional arguments:
{bootstrap,cite,demo,init,outputs,run}
Verb to execute
args Arguments for the verb
options:
-h, --help show this help message and exit
--version Print version information
These are the most common sorcha verbs:
init Initialize a new simulation
run Run a simulation
outputs Manipulate/package sorcha outputs
demo Set up a demo simulation
bootstrap Download datafiles required to run sorcha
cite Outputs the citation to a file
To get more information, run the verb with --help. For example:
sorcha run --help