Ultimate space simulation software

 
User avatar
Mosfet
World Builder
World Builder
Posts: 974
Joined: 24 Oct 2016
Location: Italy

The Gaia Mission

25 Apr 2018 10:08

After an exciting presentation of the release 2, whoever has a VR headset probably would like to try this application, updated just today:

Gaia Sky 2.0.0
https://zah.uni-heidelberg.de/institutes/ari/gaia/outreach/gaiasky/

Gaia Sky VR
https://github.com/langurmonkey/gaiasky/blob/vr/README.md

Small color map of measured stars - Release 2:
Image

http://sci.esa.int/gaia/60198-gaia-hertzsprung-russell-diagram/
Image
"Time is illusion. Lunchtime doubly so". Douglas N. Adams
My mods | My specs: Asus x555ub - cpu i5-6200u, ram 4gb, gpu nvidia geforce 940m 2gb vram | IRC freenode.net canale ##SpaceEngineITA
 
User avatar
FastFourierTransform
Explorer
Explorer
Topic Author
Posts: 197
Joined: 17 Nov 2016

The Gaia Mission

25 Apr 2018 16:14

Image

Today we had a very busy day here at the European Space Agency (ESAC center) in Madrid, as you can imagine. It has been a mind-bending experience for me. I'm very tired right now and tomorrow we would have to work much more, so don't expect any news from my side right now. Also, we have been probably attacked so the servers of the Gaia Archive have shut down for some time.

You would have to wait for the interesting results, but for now, in the few hours that we have been able to work with the data, some incredible things have pop-up that are evident on sight;

White Dwarfs: there is direct confirmation that they are presented in 3 different types. Those are clearly visible in the HR diagrams that we saw today. So yeah white dwarfs are not just a kind of object with small differences but they have 3 entire categories that we couldn't separate previously.

The orbits of satellite galaxies of the Milky Way (and halo globular clusters) and even their rotation are now known (some papers are coming)

Someone has come up with a U-V diagram (U and V are two components of motion in the galactic frame) of the entire dataset. We now see clumps in the diagram (impossible to see with Hipparcos or DR1) that have to be interpreted as stellar populations with different motions in the disc, we can now separate them in terms of motion and understand their history with minute precision (even if their spatial distribution looks homogeneous and there's no clear link between them). Fascinating discovery of today!!: There is a layered structure that was previously unknown (someone suspected it should be there) in the U-V diagram. It has been interpreted by some of the guys in the room as a stellar density wave in the galactic disc produced by the collision with an ancient galaxy.

There are many news guys. We have opened a window to the universe today. Our knowledge is going to expand incredibly. I'm so excited. And even If I can't write too much, since work has to continue tomorrow, and the all the rigorous research it's starting now, I can tell you that this is an historic moment. The scientific community will finish digesting this information in a century. Many many things are deep inside these nearly 2 billion row tables.

Just watch this diagram:
Image
You can see how there has been an historic trend for stellar catalogs to be more and more precise and have more and more stars in it. Since the y-axis is in logarithmic scale this trend means that we are getting exponentially better and better with precision for parallaxes. But watch where is Gaia. It is totally out of trend. We should expect Gaia in a few centuries not now. But here it is. Not only is better than the previous catalogs (as expected), not only is way better than the previous catalogs (as the historic exponential growth in resolution tells us) but it is so good that has broken the trend. This is truly magnificent.

Think about the fact that the Tycho catalogue was precise enough to allow for the first determination of an elliptical orbit (that of Mars) and with it we had the first experimental milestone of the Copernican revolution and the possibility to formulate Kepler's laws. What should we expect from 1.7 billion stars? This catalogue is like opening eyes for the first time to the history and geography of our own galaxy.

News are ahead ;)
 
User avatar
Hornblower
Pioneer
Pioneer
Posts: 562
Joined: 02 Nov 2016
Location: Gale Crater
Contact:

The Gaia Mission

25 Apr 2018 17:36

Mosfet wrote:
http://sci.esa.int/gaia/60198-gaia-hertzsprung-russell-diagram/

it would be sort of interesting to know what those outlier stars are like. Particularly the ones between main sequence and white dwarfs
"Space is big. Really big. You just won't believe how vastly, hugely, mind-bogglingly big it is. I mean, you may think it's a long way down the road to the chemist, but that's just peanuts to space." - Douglas Adams
 
User avatar
midtskogen
Pioneer
Pioneer
Posts: 569
Joined: 11 Dec 2016
Location: Oslo, Norway
Contact:

The Gaia Mission

25 Apr 2018 21:54

Could Gaia shed light on the dark matter/energy questions?  I mean, it's also a mapping of the gravity of the galaxy, besides a mapping of the visible stars of the galaxy, since it has much data on the movement of stars.
NIL DIFFICILE VOLENTI
 
User avatar
Watsisname
Science Officer
Science Officer
Posts: 1184
Joined: 06 Sep 2016
Location: Bellingham, WA

The Gaia Mission

25 Apr 2018 23:30

It could definitely help with modelling the distribution of dark matter in the galaxy.  Probably not so much for dark energy, which is more relevant on a broader cosmological scale.
 
User avatar
midtskogen
Pioneer
Pioneer
Posts: 569
Joined: 11 Dec 2016
Location: Oslo, Norway
Contact:

The Gaia Mission

26 Apr 2018 04:22

Is this database something SE could handle, on a computer with sufficient memory?  My home server has 64 GB, which might be tight, I guess.  But perhaps with a fast SSD and a clever database format with different detail layers it could work.
NIL DIFFICILE VOLENTI
 
User avatar
FastFourierTransform
Explorer
Explorer
Topic Author
Posts: 197
Joined: 17 Nov 2016

The Gaia Mission

26 Apr 2018 05:56

midtskogen wrote:
Source of the post Could Gaia shed light on the dark matter/energy questions?  I mean, it's also a mapping of the gravity of the galaxy, besides a mapping of the visible stars of the galaxy, since it has much data on the movement of stars.

Yes, indeed. For dark matter an ESA press release said:
By watching for the large-scale motion of stars in our Galaxy, it will also probe the distribution of dark matter, the invisible substance thought to hold our Galaxy together.

Consider also that not only we are now capable of mapping the fine structure of dark matter in out galaxy but also in broad terms thanks to the accurate motion of globular clusters in the halo.
For the dark energy I don't have an answer. But considering the fact that DR2 includes a new catalog of 500.000 quasar detections (some are going to lensed sources which means we are probing vast distances) maybe someone can come with some ideas to further constrain cosmological models.
Another interesting topic in fundamental physics that is going to emerge from DR2 is gravitational wave detection (just by observing the changes in the path of light using the exquisite astrometry of the spacecraft).


midtskogen wrote:
Source of the post Is this database something SE could handle, on a computer with sufficient memory?  My home server has 64 GB, which might be tight, I guess.  But perhaps with a fast SSD and a clever database format with different detail layers it could work.

I don't know what sizes could SE actually reach but I can tell you something about the amount of data. Currently the DR2 main catalogue occupies 587 GB of memory (All columns for the 1.692.919.135 stars). But it is true that not all the information would be necessary for SE. I've just done a reduction of the data such that we are now only considering stars that have parallax errors of around 5% (very very nice distances for SE) and those are 148.476.074 (the 8,8% of the entire DR2). This means that we are dealing with 51,5 GB or so of data. But is also true that we don't need to know all the 96 columns for each of those stars. It is sufficient with 5 parameters for SE (Source ID, ra, dec, parallax and G magnitude) even if DR2 has Temperature and Radius for the stars also (using some models) which could be useful for SE. But yeah, taking only DR2 stars with good quality parallaxes and taking only those 5 parameters for each one we are going to deal with around 2.7 GB of data. Any computer can handle this in memory but accessing the information and making it accessible from SE in terms of milliseconds is a feat that seems impossible to me. But who knows I've saw many things in SE that I though were completely impossible for one developer. The ideal would be to have some sort of SQL capability incorporated in SE for something like this.

In the next Gaia release, the DR3 there are going to be flags marking which are binary resolved systems and maybe even some orbital parameters (something interesting for SE). The parallax measurements are going to be better since the time-baseline increases and with it the number of observations for each star, which means that we are probably having many more stars with 5% accurate parallaxes. Also is very probable that DR3 might reach the 2 billion star milestone (expanding a "little" the 1.7 billion in DR2). So this will keep growing. DR3 is expected for 2020.
 
User avatar
midtskogen
Pioneer
Pioneer
Posts: 569
Joined: 11 Dec 2016
Location: Oslo, Norway
Contact:

The Gaia Mission

26 Apr 2018 06:03

Pretty cool.  Considering that the database is 1.4 TB (though likely more verbose than strictly needed), I wonder what the size of the total observations that lead to the 1.4 TB summary might have been.
NIL DIFFICILE VOLENTI
 
User avatar
FastFourierTransform
Explorer
Explorer
Topic Author
Posts: 197
Joined: 17 Nov 2016

The Gaia Mission

26 Apr 2018 06:32

midtskogen wrote:

No, no. It's 587 GB. I'm sorry I was working with another thing and confused the numbers XD
The post is now corrected. Sorry for that
 
User avatar
FastFourierTransform
Explorer
Explorer
Topic Author
Posts: 197
Joined: 17 Nov 2016

The Gaia Mission

10 May 2018 03:48

I was going to write about some Gaia DR2 news but since I'm still quite busy and since today PBS SpaceTime released a very good account for all that is happening right now I will just add the video here :D

I will explain some of these and other very interesting things that are currently going on with DR2 data analysis in the future. But for now, this is a majestic explanation.





The movement of 1.3 billion stars in the next 800.000 years as seen from Earth:




The video was produced using GaiaSky. A free and open source software that allows for quick DR2 data visualization. The program itself is very user-friendly and allows to travel in a simulated universe just like SpaceEngine (it is uglier but it has stellar movements). Give it a try! Maybe SE could borrow some code from them (since it's open source).




The Large Magellanic cloud slowly rotating through 2 million years of history (just by using Gaia's proper motions). ¡You are watching the rotation of a satellite galaxy!




In this video (also made with GaiaSky) you can see the extension of the catalogue throughout the entire Milky Way. You can also see the arrows marking stellar motion.




And finally a video showing the 14.099 asteroids detected in DR2 so far. This is important, I'm saying "so far" because these are just the obvious asteroids, we classified them just as the data release came out using known objects from other databases. Just a few of those are actually newly discovered asteroids. Bare in mind this; there are possibly hundreds of thousands of asteroids in DR2 data waiting for a much more clever analysis. Expect thousands of new asteroid discoveries in the next months as the data is studied with much more care.

 
User avatar
SpaceEngineer
Author of SpaceEngine
Author of SpaceEngine
Posts: 662
Joined: 17 May 2016
Location: Saint-Petersburg
Contact:

The Gaia Mission

10 May 2018 07:11

SE can't handle more than few million stars right now. I will rewrite the catalog stars octree so it will support dynamic loading of visible nodes only (just like procedural stars octree works). This way it will support unlimited number of stars, just like terrain engine supports unlimited resolution of textures (almost, limited only by accuracy of floating point numbers).

Star motions is a complex topic, because octree is a static data structure. I can make a hack and expand the octree nodes ~2 times (so they will intersect each other), this way they could contain their stars even if they are moving. The limitation will be performance and accuracy: with GAIA data, we have only linear movement data, so extrapolating it for 100 million years have no sense. 1-2 million years could be limit.

To represent a star, SE need its 3D position, luminosity, and spectral class (SE must know what to render: a main sequence star, red giant or white dwarf). GAIA doesn't provides the class, but it provides radius and effective temperature for many stars. These data can be used to classify stars in SE. Or I may expand the star solver so it will make classification based on R-V color, or whatever GAIA provides.
 
User avatar
FastFourierTransform
Explorer
Explorer
Topic Author
Posts: 197
Joined: 17 Nov 2016

The Gaia Mission

10 May 2018 08:01

Those are amazing news Vladimir! Expanding the ability to load much more star would be just one of the most awesome features ever.

SpaceEngineer wrote:
Source of the post To represent a star, SE need its 3D position, luminosity, and spectral class (SE must know what to render: a main sequence star, red giant or white dwarf). GAIA doesn't provides the class, but it provides radius and effective temperature for many stars. These data can be used to classify stars in SE. Or I may expand the star solver so it will make classification based on R-V color, or whatever GAIA provides.

For DR2 we have the full 6D parameters (two sky coordinates, parallax, radial velocity and two components of proper motion) for 1.3 billion stars. Photometry is done with G, B and R bands (they have different precision). Under the "teff_val" column of DR2 you have a precomputed effective temperature for the star (done using photometry and extinction measurements) so, in principle this would be easier. "teff_val" is only available for 161.5 million stars (I don't know how many of those have also at the same time the 6D astrometric parameters but I estimate more than 90%). Radius and luminosity for the star is only available for 77 million cases. So maybe the best we can handle right now are at max 77 million stars for SE. Also, we should apply some strict criteria If we want to import part of DR2 to SE; there is a photometric quality flag "phot_proc_mode" that goes like 0: Gold, 1: Silver and 2: Bronze. Now with DR2 the vast majority of stars with photometry have high quality (Gold) data so we should keep that as a criterion (I've seen silver and bronze two-color diagrams and are just ugly). For 6D parameters there's also the issue I always talk about; the fact that the inverse of parallax is not an unbiased estimator for distance due to Lutz-Kelker bias and others. We should never go with parallaxes with 10% uncertainties or larger and even with 5% to 10% uncertainties we should perform a Lutz-Kelker correction if possible. If we wait for some people to perform the calculations we could have all the needed data for tens of millions of stars in SE. And if we wait for DR3 in 2020 this is just going to get bigger.



SpaceEngineer:Source of the post The limitation will be performance and accuracy: with GAIA data, we have only linear movement data, so extrapolating it for 100 million years have no sense. 1-2 million years could be limit.

You are absolutely right. No more than 2 million years to the past and to the future for realism. Also true, thanks to Gaia's DR2 we are going to be able to produce high quality models of the galactic potential. I don't know how hard would it be to implement a calculator for longer time intervals. Taking into account halo, disk, bulge and spiral arms you could easily extend the time-frame to hundreds of millions of years. Even with a simple disk potential model you could make an approximation for stellar orbits around the Galaxy. But yeah this is just wild speculation right now. Your post made me so happy.



P.D: I'm very busy right now so I have to apologize to you because I never finished the corrected stellar catalogue for SE that we talked about, but I'm thinking and rethinking it (now with more precise data from DR2) so I hope to make it in the summer at worst.

Who is online

Users browsing this forum: No registered users and 1 guest