Gitea hosted repository has become inaccessible

rpgoldman

I have a repository on Opalstack that I have been able to push to in the past, but that when I try to interact with now gives me:

fatal: 'rpg/web-page.git' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

I'm mystified because I don't believe that I have changed any settings in the 7 months since I last pushed to this repo successfully. However, if it helps, I believe I have a new laptop since then.

I checked the ssh interaction by running GIT_SSH_COMMAND="ssh -vvv" git pull and it appears that my SSH key authenticated successfully:

debug1: Server accepts key: /Users/rpg/.ssh/id_rsa RSA SHA256:55Tmwa58PAlApi9E/vgaus6Fo2Gx9AEN+Ef0JBbAAOg explicit agent

I'm not sure how to interpret the SSH output, but here is what I see at the end of my attempt to pull:

debug1: Sending command: git-upload-pack 'rpg/web-page.git'
debug2: channel 0: request exec confirm 1
debug3: send packet: type 98
debug2: channel_input_open_confirmation: channel 0: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug2: channel 0: rcvd adjust 2097152
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: exec request accepted on channel 0
debug3: receive packet: type 98
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug3: receive packet: type 98
debug1: client_input_channel_req: channel 0 rtype eow@openssh.com reply 0
debug2: channel 0: rcvd eow
debug2: chan_shutdown_read: channel 0: (i0 o0 sock -1 wfd 6 efd 8 [write])
debug2: channel 0: input open -> closed
debug2: channel 0: rcvd ext data 65
debug3: receive packet: type 96
debug2: channel 0: rcvd eof
debug2: channel 0: output open -> drain
debug3: receive packet: type 97
debug2: channel 0: rcvd close
debug3: channel 0: will not send data after close
debug2: channel 0: obuf_empty delayed efd 8/(65)
fatal: 'rpg/web-page.git' does not appear to be a git repository
debug2: channel 0: written 65 to efd 8
debug3: channel 0: will not send data after close
debug2: channel 0: obuf empty
debug2: chan_shutdown_write: channel 0: (i3 o1 sock -1 wfd 7 efd 8 [write])
debug2: channel 0: output drain -> closed
debug2: channel 0: almost dead
debug2: channel 0: gc: notify user
debug2: channel 0: gc: user detached
debug2: channel 0: send close
debug3: send packet: type 97
debug2: channel 0: is dead
debug2: channel 0: garbage collecting
debug1: channel 0: free: client-session, nchannels 1
debug3: channel 0: status: The following connections are open:
  #0 client-session (t4 r0 i3/0 o3/0 e[write]/0 fd -1/-1/8 sock -1 cc -1)

debug3: send packet: type 1
debug1: fd 0 clearing O_NONBLOCK
debug3: fd 1 is not O_NONBLOCK
Transferred: sent 3872, received 3064 bytes, in 0.6 seconds
Bytes per second: sent 6360.1, received 5032.9
debug1: Exit status 128
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

I can't tell from that if this is an SSH problem or a git problem.

Interestingly, the key that was accepted to start the SSH transaction is the one that is configured for my ssh shell account, id_rsa and is not the same key that I configured in the Gitea UI, which is gitea_git_goldman-tribe_org_rsa. Could this be the problem? I added id_rsa.pub to the Gitea UI, and that did not fix the problem: same error.

On opalstack I did this:

$ pushd /tmp
/tmp ~/apps/gitea/repos/rpg/web-page.git ~
[rpg@opal11 tmp]$ git clone ~/apps/gitea/repos/rpg/web-page.git
Cloning into 'web-page'...
done.

So as far as I can tell, there is nothing wrong with the repository itself, since I can clone it locally.

I believe that means there is something wrong in trying to operate on this repository through gitea, but at that point I'm lost.

I don't recall doing anything to modify the contents of the repositories on Opalstack, so I am at a loss to explain this. I don't understand how to diagnose the problem, either. Is there something I can do on Opalstack to check and see if the prop

sean

rpgoldman try removing all of your keys from Gitea, and ensure that they are also removed from ~/.ssh/authorized_keys on your server then try accessing the repo with your Gitea user password. If that works, then try adding the Gitea SSH key back to Gitea.

Do not add your regular SSH key to Gitea as it will break your ability to log in to the shell. If you've already done this and aren't able to access the server then please email support for assistance.

sean

rpgoldman also please review https://docs.opalstack.com/topic-guides/git/#using-ssh-keys-with-gitea to ensure that you've got your key configured correctly on your end.

rpgoldman

Sorry to bother you, but I'm still a bit stuck. I did the following:

Remove all the ssh keys from gitea
Remove the configuration stanza for my gitea site from ~/.ssh/config
Move ~/.ssh/authorized_keys to ~/.ssh/authorized_keys.bak on my opalstack site.

I can still log in to my gitea web site using the password for my gitea user account (not my shell account).

But when I try to git push to gitea, it still fails for me:

$ git push
redacted@git.goldman-tribe.org's password:
Permission denied, please try again.
redacted@git.goldman-tribe.org's password:
Permission denied, please try again.
redacted@git.goldman-tribe.org's password:

This is with the same password that allows me to successfully log in to Gitea.

When I use my shell user password instead, I get this again:

$ git push
rpg@git.goldman-tribe.org's password:
fatal: 'rpg/web-page.git' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

This is for my git repo URL: <redacted>@git.goldman-tribe.org:<redacted>/web-page.git

So I guess I don't understand how the git access works under Gitea. I take it that git@ is interpreted to be ssh access, correct? Then given that git.goldman-tribe.org is the same host as the host I use for ssh shell access, how does it know to use Gitea authentication instead of shell authentication?

Indeed, it looks kind of like what might be happening is:

When I connect with my shell user PW I get in, but my shell user doesn't have access to the repo but
When I connect with my Gitea PW I don't get in through ssh, but presumably I would get access to the repo.

In that documentation page it says:

At this point you should be able to access your repos over SSH with no password with your Gitea app's shell user.

I'm not sure what "your Gitea app's shell user" means. Does it mean my opalstack ssh shell user? Or the Gitea user account I have created in the Gitea web interface? Should that just be "the Gitea user account you created in the Gitea web application"?

One thing that might be significant: the Gitea user id I created has the same name as my shell user. Is that A Bad Thing?

sean

So I guess I don't understand how the git access works under Gitea. I take it that git@ is interpreted to be ssh access, correct?

When you access a Gitea git repository via SSH, you must make the connection using the name of the shell user that owns the Gitea application.

For example, if you click the "SSH" button in Gitea to get to your repo's SSH URL, you'll see something like shell_username@domain.com:gitea_repo_owner/name_of_repo.git.

shell_username@domain.com will be the same for every repo hosted on that Gitea instance, regardless of who owns the repo or who is accessing it.

Github works the same way - in their system the user on their end is "git" which is why every Github SSH URL uses git@github.com.

Then given that git.goldman-tribe.org is the same host as the host I use for ssh shell access, how does it know to use Gitea authentication instead of shell authentication?

It knows this via the key configuration covered in step 2 of our instructions, but since it works by associating the Gitea key with your Gitea app's subdomain you will not be able to use that same subdomain for normal interactive SSH sessions.

I recommend you use your git subdomain only for Gitea access and some other subdomain (or your server's canonical hostname opalN.opalstack.com) for regular SSH access.

rpgoldman

Great. I'm sorry to be dense, but it's a little odd, so I am going to read this back to you.

I should have a shell user with an SSH key shell_id_rsa.
I should have a gitea user with an SSH key gitea_id_rsa.
I set up .ssh/config so that it has one stanza for opal<n>.opalstack.com and one stanza for git.goldman-tribe.org, my gitea server IP name.
I dump out gitea_id_rsa.pub and put that in the ssh user page for gitea.

Indeed, this almost works. But now I have a new error:

rpg@rpg-macbook-pro: ~/personal-web-page $ git push
Remote "origin" does not support the Git LFS locking API. Consider disabling it with:
  $ git config lfs.https://git.goldman-tribe.org/rpg/web-page.git/info/lfs.locksverify false
batch request: Gitea: Unknown git command: exit status 1
error: failed to push some refs to 'git.goldman-tribe.org:rpg/web-page.git'
rpg@rpg-macbook-pro: ~/personal-web-page $ git config lfs.https://git.goldman-tribe.org/rpg/web-page.git/info/lfs.locksverify false
rpg@rpg-macbook-pro: ~/personal-web-page $ git push
Uploading LFS objects:   0% (0/1), 0 B | 0 B/s, done.
batch request: Gitea: Unknown git command: exit status 1
error: failed to push some refs to 'git.goldman-tribe.org:rpg/web-page.git'

Is gitea compatible with git-lfs?

Another problem is that I am no longer able to use ssh connections:

$ slogin opal11.opalstack.com
PTY allocation request failed on channel 0
Hi there, rpg! You've successfully authenticated with the key named laptop gitea key, but Gitea does not provide shell access.
If this is unexpected, please log in with password and setup Gitea under another user.
Connection to opal11.opalstack.com closed.

I have separate .ssh/config stanzas for opal11.opalstack.com and for git.goldman-tribe.org but it seems that the stanza for the second is always used. Is that because ssh is matching based on IP address instead of IP name?

If so, is this what the message about setting up gitea as another user is about? Does that mean add a new shell user account and use that account as the owner of gitea?

If so, maybe the manual should say always to install gitea that way (maybe it does and I missed it).

Thanks,
R

sean

rpgoldman Is gitea compatible with git-lfs?

Gitea supports git-lfs but it is disabled by default: https://docs.gitea.io/en-us/git-lfs-setup/

rpgoldman Another problem is that I am no longer able to use ssh connections:

I've checked your setup on the server and the only key that you have installed is for Gitea. You can see this by looking in ~/.ssh/authorized_keys in your shell user home dir, the key will be prefixed with a gitea command like this:

# gitea public key
command="/home/xxx/apps/yyy/gitea --config=...

This means that any connection that uses that key is going to run Gitea's client instead of a normal shell.

In my own testing with multiple stanzas I did not have any problems with the wrong key being used, eg with the following local SSH config...

Host opalX.opalstack.com
    IdentityFile ~/.ssh/id_rsa
Host gitea.xxx.opalstacked.com
    IdentityFile ~/.ssh/gitea_rsa
    IdentitiesOnly yes

... logins to opalX.opalstack.com use the normal key id_rsa while logins to gitea.xxx.opalstacked.com pick up gitea_rsa, even though both names resolve to the same IP.

rpgoldman If so, is this what the message about setting up gitea as another user is about? Does that mean add a new shell user account and use that account as the owner of gitea?

No, that should not be necessary if your key configuration is correct.

Feel free to email your local ~/.ssh/config to our support team if you'd like us to look it over.