Feeds:
Posts
Comments

Archive for the ‘Networking’ Category

Bagi anda yang menggunakan Cisco router, mungkin suatu saat anda akan mengalami hal ini atau anda pernah mengalaminya. 3 hari yang lalu saya mengalami hal tersebut. Tiba-tiba salah satu router kami mengalami masalah, ping putus2 sekalipun ping ke ip loopback, SSH juga susah sekali masuk setelah mencoba beberapa kali akhirnya muncul halaman login, tetapi authentikasi gagal karena koneksi ke TACACS server juga ikut terganggu. Cukup merepotkan juga apalagi saya hanya diberi waktu 30 menit untuk melakukan analisa dan mencari solusinya. Sedikit informasi saya menggunakan router Cisco 7600.

Pada menit pertama saya berhasil mengidentifikasi permasalahannya, yaitu Utilisasi CPU yang terlalu tinggi mencapai 99%”. Hal ini saya ketahui melalui monitoring tool yang saya gunakan, jadi saya gak perlu repot2 login ke router karena dengan kondisi seperti sekarang sangat sulit untuk bisa masuk ke router. Setelah tau penyebabnya maka sekarang saya harus mencari penyebab “High CPU Utilization”. Berikut hal-hal yang dapat anda lakukan apabila mengalami problem yang sama.

Identifikasi Masalah

SHOW PROCESS CPU NOTIFICATIONS (if any)

    Router#show process cpu sorted

      Dengan command ini anda akan mendapatkan informasi tetang CPU utilisasi, process-process yang menggunakan CPU, dan interrupt percentages. Informasi tersebut bisa anda dapatkan pada baris pertama dari output show.

      router#sh proc cpu sort

      CPU utilization for five sec: 99%/54%; one minute: 99%; five minutes: 99%
      PID ntime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
      77 11697800 822858 14216 38.52% 37.51% 37.71% 0 IP Input
      128 702920 714062 984 2.47% 2.57% 2.50% 0 Tag Input

      Keterangan:

      CPU utilization for five seconds: x%/y%; one minute: a%; five minutes: b%
      Total CPU Utilization: x%
      Process Utilization: (x - y)%
      Interrupt Utilization: y%
      Process Utilization is the difference between the Total and Interrupt (x and y). The one and five minute utilizations are exponentially decayed averages (rather than an arithmetic average), therefore recent values have more influence on the
      calculated average.
      
      

      SHOW LOG

      Lakukan “show logging” untuk mendapatkan informasi log router, barangkali ada informasi berguna yang dapat membantu untuk melakukan identifikasi masalah dan menemukan akar permasalahannya.

      Dampak dari High CPU Utilization

      – Input queue drops

      – Slow performance

      – Slow response int Telnet or unable to Telnet router

      – Slow response on the console

      – Slow or no response to ping

      – Router doesn’t send routing updates

      Kemungkinan Penyebab High CPU Utilization

      Berdasarkan hasil “show process cpu” saya mendapatkan informasi bahwa IP Input menggunakan resource CPU cukup besar. Maka ada kemungkinan penyebabnya adalah seputar IP Input. Tetapi meskipun demikian, saya tidak menutup mata akan adanya penyebab yang lain. Hal-hal yang mungkin menyebabkan High CPU Utilization antara lain:

      Hardware failure

      Perkiraan saya untuk hardware failure yang dapat menyebabkan High CPU Utilization adalah RSP failure dan/atau VIP failure.

      Configuration

      Berdasarkan hasil “show process cpu” kemungkinan permasalahan berhubungan dengan IP Input, jadi bisa saja karena konfigurasi fast switching, TCP intercept, penggunaan IP NAT, dll. Atau bisa juga karena ada yang melakukan DoS.

        Langkah-langkah Penanganan

        Hardware failure

        a. RSP Failure

        Jika yang bermasalah RSP ya diganti aja :D. Tetapi sekedar saran mungkin opsi ini dijadkan pilihan terakhir saja, karena untuk menggangti RSP router harus dimatikan. Total estimasi downtime yang dibutuhkan kurang lebih 15 menit. Waktu yang dibutuhkan router 7600 untuk booting kira2 10-15, itu termasuk update routing, dll.

        b. VIP Failure

        Untuk memastikan apakah yang bermasalah VIP atau bukan, maka anda bisa melakukan:

        Router#sh controllers vip all proc cpu | i utilization

        CPU utilization for five seconds: 1%/1%; one minute: 1%; five minutes: 1%

        CPU utilization for five seconds: 5%/5%; one minute: 5%; five minutes: 5%

        CPU utilization for five seconds: 21%/21%; one minute: 21%; five minutes: 21%

        CPU utilization for five seconds: 19%/19%; one minute: 20%; five minutes: 19%

        Jika anda ingin melihat detil prosesnya maka lakukan perintahsh controllers vip vip_number proc cpu “.

        Configuration

          TRY THIS: If IP Input is consuming the CPU, one of the following might be the cause:

          Fast switching is disabled on an interface (or interfaces) that has a lot of outgoing traffic. Examine the output of the ‘show interfaces switching’ command to see which interface is burdened with traffic. Re-enable fast switching on that interface.

          TCP Intercept is enabled. TCP Intercept requires process switching for all packets during session set-up.

          Fast switching is disabled on an interface which supports more than one network and is routing traffic between them. This can occur when an interface has one or more secondary network addresses configured.

          INFO: The router will process switch all packets sourced from the interface and destined to host(s) off the same interface which is a CPU-intensive task. Use the ‘ip route-cache same-interface’ interface configuration command to allow packets to be fast switched on the same interface.

          Traffic that can’t be fast switched is arriving. This could be any of the following types of traffic:

          * Packet for which there is no entry yet in the switching cache.

          INFO: If there is a device in the network which is generating lots of packets at an extremely high rate for devices reachable through the router and is using different source or destination ip addresses, there won’t be a match for these packets in the switching cache, so they will be processed by the IP Input process. This source device can be a malfunctioning device or a device attempting a Denial-of-Service (DOS) attack.

          * Packets destined for the router (ie. Routing Updates or a Spoof Attack)

          * IP packets with options

          * Packets that require protocol translation

          * Multilink PPP

          * Packets that require policy routing.

          INFO: IOS versions 11.3 and higher allow policy-routed packets to be fast switched. Usee the ‘ip route-cache policy’ interface configuration command to allow policy-routed packets to be fast switched.

          * Packets going through serial interfaces with X.25 encapsulation. In the X.25 protocol suite, flow control is implemented in layer 2 of the OSI model.

          * Compressed traffic. If there’s no Compression Service Adapter (CSA) in the router, compressed packets must be process-switched.

          * Encrypted traffic. If there’s no Encryption Service Adapter (ESA) in the router, encrypted packets must be process-switched.

          A lot of packets, arriving at an extremely high rate, for a destination in a directly attached subnet, for which there is no entry in the ARP table. This shouldn’t happen with TCP traffic, because of the windowing mechanism, but it can happen with UDP traffic.

          A lot of multicast traffic going through the router. Unfortunately, there’s no easy way to examine the amount of multicast traffic. If you’ve configured multicast routing on the router, you can enable fast switching of multicast packets using the ‘ip mroute-cache’ interface configuration command (fast switching of multicast packets is off by default).

          A lot of broadcast traffic. Check the number of broadcast packets in the ‘show interfaces’ command output.

          Too much traffic is passing through the router. If the router is over-used and is incapable of handling this amount of traffic, try distributing the load among other routers or consider purchasing a high-end router.

          IP NAT is configured on the router and there are lots of DNS packets going through the router. UDP or TCP packets with source and/or destination port 53 (DNS) are always punted to process level by NAT.

          Check who’s logged on to the router and what they are doing. If someone is logged on and is issuing commands that produce long output, the high CPU utilization by the IP input process will be followed by a much higher CPU utilization by the virtual EXEC process.

          Make sure all debugging commands in your router are turned off by issuing the undebug all or no debug all command.

          Check for a possible security issue. Commonly, high CPU utilization is caused by a security issue, such as a worm or virus operating in your network. Usually, a configuration change, such as adding additional lines to your access lists can mitigate the effects of this problem. Check the Cisco Product Security Advisories and Notices for information on the most likely causes and specific workarounds.

          Read Full Post »

          Cisco-MRV Comparison

          This is comparison between Cisco 3560 and MRV 3234. Hopefully it can help you to decide which equipment match with your needs. Table Comparison

          Read Full Post »

          Configure router to pass DHCP requests from local clients to a centralized DHCP server

          The traditional role of routers in DHCP has been simply to act as a proxy device, forwarding information between the client and server. Since IOS level 12.0(1)T, Cisco routers also have DHCP server and client features. But the DHCP proxy function is still the most common for routers.

          Because the initial DHCP request comes from a client that typically doesn’t have an IP address, it must find the server using a Layer 2 broadcast. So, if the router was not able to function as a proxy for these broadcasts, it would be necessary to put a DHCP server on every network segment.

          IP Helper Configuration Example:

          Router1#configure terminal 
          Enter configuration commands, one per line.  End with CNTL/Z.
          Router1(config)#interface Ethernet0
          Router1(config-if)#ip helper-address 172.25.1.1
          Router1(config-if)#ip helper-address 172.25.10.7
          Router1(config-if)#exit
          Router1(config)#end
          Router1#
          NB: 172.25.1.1 and 172.25.10.7 is DHCP server IP Address

           

          The DHCP server needs two critical pieces of information before it can allocate an IP address to the client. (more…)

          Read Full Post »