You will need to update your LD_LIBRARY_PATH, so that it points to the /usr/local/cuda-9.0/lib64.
Add the following line to your .bashrc file (or any other terminal you use)
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-9.0/lib64/
The Tensorflow Binaries do not support Cuda 9.1 as of now.
So the solution might be just installing Tensorflow from source.
I too had the above mentioned issue while I had installed Cuda 9.1+cuDNN 6 for Tensorflow and this is what worked for me.
https://devtalk.nvidia.com/default/topic/1026198/cuda-setup-and-installation/cuda-9-0-importerror-libcublas-so-8-0
Additionally, you have the following two options:
CUDA 8.0 + Tensorflow – the most stable release of CUDA that works with Tensorflow Binaries.
CUDA 9.0 + Tensorflow – latest CUDA version that Tensorflow Binaries are compatible with.