some more information about your host OS and its network structure could be useful...
generally, you need a ssh server installed on the host: IIRC its not installed by default in Debian for example, there you would apt-get install openssh-server (and install that on the guest too...)
OTOH you need a network connection other than VBox default NAT. If -for example again- your host is connected to an ethernet LAN you would select "bridged" for the guest