[Python] Converting a dataset to COLMAP structure

My goal is to use COLMAP to obtain a point cloud from a set of images with known camera information, the way it is described here: https://colmap.github.io/faq.html#reconstruct-sparse-dense-model-from-known-camera-poses. Ultimately, I want to construct a dataset that is structure like the COLMAP output, but with a priori known camera information, and use it to run 3D Gaussian Splatting.

I am using the DTU dataset (https://github.com/lioryariv/idr/blob/main/DATA_CONVENTION.md). Each image is assigned a 'world_mat' and a 'scale_mat' matrix.

I do the following:

P = (world_mat @ scale_mat)[:3, :]
K, R, t = cv2. decomposeProjectionMatrix(P)[:3]

fx = K[0, 0]
fy = K[1, 1]
cx = K[0, 2]
cy = K[1, 2]

t = (t[:3] / t[3]).reshape(3)
qx, qy, qz, qw = Rotation.from_matrix(R).as_quat()


then format this into cameras.txt and images.txt, run COLMAP, etc.

However, the results I get from 3DGS are terrible. I am pretty much sure there is something wrong with how I obtain K, R, t or with how I transform them to the COLMAP format.

I also tried transposing the rotation matrix, negatig the translation vector (pretty much everything I could think of or read about somewhere), but nothing really works.

Continue reading...