There are several places to install the EGT probe. However, EGT readings will vary depending on where the EGT probe is mounted, as pre-turbo EGTs are always higher than post-turbo EGTs.
Installing the EGT in the exhaust manifold is the preferred method. It is also one of the easier methods, since the manifold is easily accessible.
Although the 'traditional' manifold mounting point has been cylinder #1, recent information from Team Rip Engineering shows that the #2 cylinder is the hottest of the four. Recommendations for the distance between the probe and cylinder head range from 1 to 6 inches.
Another possibility is to install the probe in the manifold 'collector' - the place where all the runners converge. The idea here is to get an average reading of how all four cylinders are running, rather than a specific reading on one cylinder. Some people believe this is safer since you can monitor the entire engine. There is a good write-up on the difference between runner and collector placements here, courtesy of Steve Wells, that can help you decide between the two.
Other possibilities include the oxygen sensor housing and the downpipe. Diamond-Star Specialities doesn't recommend the oxygen sensor housing because the temperature drop between the manifold and the housing varies from car to car. Read about this here.
For downpipe installations you don't need a compression fitting, since a small exhaust leak at that location really doesn't matter. Also, if you accidentally drill a hole in the wrong location, it is more easily repaired. Unfortunately, the downpipe is fairly far removed from the engine, and only readily accessible from underneath the car. One solution is to have the probe pre-installed when upgrading or replacing the downpipe.