Running the ua.-top level domain in times of war
The network in Ukraine has so far proven to be resilient – where infrastructure has not been completely destroyed. Nevertheless, backups are being worked on.
- Monika Ermert
Dmitry Kohmanyuk has been working for the Ukrainian ccTLD .ua since its foundation. In recent years he worked as the national ccLTD’s strategist, working on the monitoring and introduction of new technology, like for example DNSSEC. Caught outside of his country on February 24, Kohmanyuk cooperates with his colleagues at home to keep the registry for half a million .ua sites running.
The company is also operating the gov.ua TLD, a much sought for and much attacked zone. Kohmanyuk connects with international partners and he tries to assure the .ua team dispersed over the Ukraine and beyond. Being "like a mum" to those trapped in Kiev felt strange and difficult, as he wanted to cry himself, he said in a chat with heise online. For now, the registry still has about 50 percent of their infrastructure in the Ukraine. But the .ua itself could become a refugee, operated from the outside.
heise online: Dmitry, how are you?
Dmitry Kohmanyuk: I try not to think about myself too much. But I am fine. I am 51 years old. I have lived my life. I left Soviet-Union Ukraine for some years to live in California for some time. After that, I went back to Ukraine. I am happy, I have seen some parts of the world. I am very pessimistic about the situation now. Some damage is reparable. We can move some things around. We can very easily move data around. But we can hardly move people around. History, especially, we cannot move around. There are things you can’t repair, there is damage that cannot be undone. All of the so called Russian-Ukrainian friendship is gone. It will never come back. Or perhaps it can come back, in a century. It will be similar to the Holocaust. People don’t forget the Holocaust. I am thinking about this a lot. I still try to be polite when i speak to people from Russia. In fact a Russian friend has invited me to stay with him, i had visited him when the war broke out.
heise online: Where are you?
Dmitry Kohmanyuk: Let’s just say I am in Eurasia. We arrived here the 14th and the plan was to go back 21. But we extended our stay for another week and then the war started. Now I am a refugee. I have no home to go back. I do have my salary from Ukraine. I hope I can be paid next week. I can survive on my savings from a few months. I am ok.
Did you consider going back?
No, I do not intend to go back now.
Did you expect the war?
I was denying it, but friends in the US and the US embassy told me that it would happen. Still I was in denial, like when you are told that somebody has cancer. You deny that he will die.
How is the .ua team?
The ccTLD team, my team, are 20 people. I have to admit, we were not ready. We were not planning for an attack and did not have a lot of backup systems in place. We had to make do and it is all right. We did the necessary things. We basically evacuated the production computers to the cloud. I will not tell you names. But we found a number of international companies. I relied on connections from international meetings like ICANN and RIPE. I started messaging people. And some got back really immediately. I decided to go to the guys in the blue suits and that did work out, better perhaps that turning to the crowd. The truth remains we were not prepared. President Zelensky said publicly that we would not be attacked. But in the end we did have enough resources, people and computing, too, to make it work.
What is the most important?
A, making the domains work, B, making the domain registration work. This is our job, it is what is most important. We are the registry that records who owns which domains. We have registars, they sell domains inside and also outside the country.
Do you still see new registrations come in at this moment?
What are people registering during these times?
I am not tracking this and I should not tell. That is private. I am just the mechanic. So the registry works, but we just had to move that away from Ukraine. Then we have the public facing part, our website and our email system. We kept that in Ukraine for the moment. For our communication we changed at messengers anyway, as the phone system is down due to outages. We also operate the zone of the gov.ua. That part we also had to move abroad, but luckily as we already had DNS servers abroad, that was easy. Now we have a distribution of 50 percent abroad and 50 percent in Ukraine. But because I knew everything in Ukraine was at a risk. I asked for additional secondaries in the cloud. So we had the A systems, the systems that were in place. Then we built the B systems, new systems in different places. And then there are the C systems, that is what we rented – or got it for free. We ended up renting quite some in the cloud. I should not mention company names, but I want to mention Cloudflare because they reached out to us proactively and offered help. We took their help and we relied on them immensely and I really want to express my gratitude to the leadership and the team there. They know who they are. All worked a lot. It started by the way when there was a denial of services on the government websites just a few days before the start of the war. And from there we had that direct link to the management there. We still have a lot of stuff in Ukraine, for sure. There are also things like accounting papers. First and foremost, we have our people there. And the people, they cannot easily relocate. Some people don’t have money, cash or cars to relocate. Also, it is only women who can go. The men cannot just leave the country or relocate. They are not allowed. When they move they have to notify the authorities.
But the technical operators can still do the work?
Still the state can draft any person of our company to the war. We have no exclusive status at all. The do not draft lots of people. We do not have a single person in the company that has been in the military, except our director who has been to the armed services. Certainly I went to school in Ukraine, during the time still part of the Soviet Union. We were shown how to fire a Kalashnikov, but there was just one Kalashnikov for the whole class, so it was a rather theoretical exercise during which we had to pretend how we would fight. Later I had a military camp in Kherson for 2 weeks. That is about all of the military training, I have.
Can you describe your workday? How do you coordinate with the team back in Kiev?
They are not all in Kiev. Some are, others not. And some people have moved around. I do not want to disclose how they are dispersed.
How many staff members left Ukraine?
Two women currently are working from outside of Ukraine.
It could be good to have some staff members outside the Ukraine, could it not?
I agree. But there is the difficulty of money transfers out of the country. It is currently prohibited.
For organizing our work we use email, we use Signal chat a lot. We are getting rid of Telegram, which we used before the war a lot for chatting, basically for security reasons. Beside the crypto we like it that nothing is archived. We have been cooperating and working online and remote since the war started.
For now the .ua registry is operative. Our bank works. We can sell our service. Getting free stuff as a donation from the outside was nice. The team helps out in some instances. When our boss had to move some servers somewhere, he used his own credit card and called it his contribution to the company. I did the same and paid for a few things. It is not long-term strategy. We are just surviving.
There has been an announcement that the Ukrainian government wants to move more of its IT infrastructure out of country, can you confirm that and how does it affect the ccTLD?
We don't take computers and move them around. We rent. We go abroad and rent servers there. For example we rent 10 servers in the EU and then turn off five at home. Except, we have not turned any of our servers off in Ukraine so far. All are still running. We just copy files and set up mirrors of our systems. Moving around hardware under this threat, would not work.
What is the biggest threat to registry?
The biggest threat to the registry is the threat to Ukraine. The military threat. The bombing, the nuclear threat. The biggest threat is the collapse of the economy and the country. And there is another thing. People are exhausted, they are tired, they are hungry. People are having mental breakdowns, i have a lot of calls and try to calm them. I feel I suddenly have to be like a mum, only i am not good at calming people. And I am a kid to my partner her at my sanctuary. I am grateful she is here.
What could help to keep the registry up?
We have solved that for the time being, I think. Our ccTLD is small. Its half a million domains currently. We can run things light-weight.
The help we need is the help to our country. Ukraine needs humanitarian aid, military aid, political aid. We need strict sanctions from businesses. Of course, you cannot win a war by not selling a PlayStation. But you know, people do what they can. I should say I am thankful for the German government for military aid, sanctions and the help to refugees.
You said, you were critical about the notion of neutrality of infrastructure? Should The ICANN and Ripe have followed the appeal from Ukraine and cut Russia from IP resources and thrown their ccTLD .ru out of the authoritative root zone?
You know, I don’t know what they should do. I know what they should not do. They should not ignore the call. I understand that they allocated 1 million to keep the Internet running in Ukraine. I do not know how that money will be allocated, but I see the effort. I personally think they could sanction their Russian partners, like other US companies have done. Apart from what US laws say, every company can decide for itself to not serve some customers. They can decide to not do business with, let’s say North Korea. They could add Russia to that list.
Should the Internet community start to use BGP filters or filter specific domains?
I put my name to a letter which asks the community to consider a list of highly targeted sanctions. Bill Woodcock distributed it. And while I was not involved in the drafting as I felt I am partial, I did approved it. The letter explains how sanctions should be done in the internet age. I think it is a good start. It might not be the final thing possible. Some things might still be found inadequate. It is a mild letter, there are no threats in there. It has no negative language. And it also talks about the humanitarian aspect of sanctions. For me, the world exists because of trade, so stopping trade hurts people. Like when Ukraine stops producing wheat, people starve. We risk economic collapse with disruption of trade, that is clear.
What about hacking attacks?
I think as long at it targets the military or the government, it is ok. Remember changing a website does not hurt. If the website of the Kremlin says the truth about the war in Ukraine, that is not criminal. I myself do not intend to involve myself in hacking attacks. I grew up with an Internet philosophy that is peaceful and non-miliatary.
Did you see attacks on .ua?
Every day and we have had an outage for a few hours just yesterday. Even Cloudflare’s systems who are resilient are not built to withstand war attacks.
You met with Russian colleagues over the years at many operator meetings. What do you say to your Russian colleagues now?
I am still not be angry at them. Most of my Russian friends from the RIPE community are still my friends. Some are not. But a lot of friends that are Russian are living abroad. What do I want to say to them? Do the right thing. And they know what that is. For the Russian and Belarussian military there are three options. They can die in battle. They can go back to their countries, that might mean they go to jail. Or they can surrender. If they surrender, they will live. So surrendering looks to be the best option – and not bombing the cities. If you pilot an airplane you still have to press that button to bomb a city. Don’t do it. Refuse orders. Surrender.
Could the .ua registry be exiled completely, could it become a refugee itself?
Yes. We continue operating no matter what. We would never disband. We do what ever it takes. We will never surrender. I just hope we win. There is no plan B.